Run in CI

@wirebench/cli (binary wirebench) runs the requests already saved in a project from a terminal or a pipeline, and turns the result into an exit code and a report a CI system understands. Four recipes get there — a GitHub Action, a GitLab template, a container image, or plain npx — all backed by the same published package.

Secrets are the caller's

None of these recipes holds or asks for a secret. Map each one your pipeline already has to a WIREBENCH_SECRET_<NAME> variable in the CI system’s own environment or variables — the same name wirebench secrets list reports — and the runner’s own masking hides the value everywhere a report could show it.

GitHub Actions

- uses: wirebench/wirebench/action@v5.0.0
  with:
    project: ./api-tests
    env: staging
    junit: reports/wirebench.xml
  env:
    WIREBENCH_SECRET_BILLING_PASSWORD: ${{ secrets.BILLING_PASSWORD }}

The action’s first step installs Node 24 (node-version, default 24) — this changes which node later steps in the same job see, so pin node-version or give the action a job of its own if a later step needs a different version. It then runs @wirebench/cli through npx and exposes the CLI’s exit code as the exit-code output, so a caller using continue-on-error: true can still branch on the result.

Every input is passed through env:, never interpolated into the action’s own script, so a value containing shell metacharacters can’t be reinterpreted as source.

Input	Default	Maps to
project	(required)	<path>
env	—	--env
select	—	selectors, one per line
vars	—	--var, one key=value per line
junit / json / html	—	--reporter junit=… etc.
bail, require-assertions, insecure	false	the flags
timeout, sla	—	--timeout, --sla
version	this action’s own ref when it is a version tag (e.g. v5.0.0 → 5.0.0), else latest	@wirebench/cli version run via npx
node-version	24	actions/setup-node

See the action’s own README for the full reference.

GitLab CI

include:
  - remote: https://raw.githubusercontent.com/wirebench/wirebench/v5.0.0/templates/gitlab/wirebench.gitlab-ci.yml

api-tests:
  extends: .wirebench-run
  variables:
    WIREBENCH_VERSION: '5.0.0'
    WIREBENCH_PROJECT: api-tests
    WIREBENCH_ENV: staging

.wirebench-run runs the CLI from ghcr.io/wirebench/wirebench-cli:${WIREBENCH_VERSION} and collects WIREBENCH_JUNIT (default wirebench-junit.xml) as a JUnit report with artifacts: when: always, so results still show up on a red pipeline.

Pin the version

The template file is not rewritten when a release tag is cut — the tagged file has to equal the tagged commit — so its own WIREBENCH_VERSION default is latest. Always set WIREBENCH_VERSION explicitly, the way the example above does, rather than relying on that default.

Map a secret to WIREBENCH_SECRET_<NAME> under Settings → CI/CD → Variables, with “Mask variable” turned on. WIREBENCH_ARGS carries anything else — extra flags, selectors — through to the CLI unchanged.

Docker

docker run --rm -v "$PWD:/work" \
  -e WIREBENCH_SECRET_BILLING_PASSWORD \
  ghcr.io/wirebench/wirebench-cli:5.0.0 run ./project --env staging --reporter junit=reports/wirebench.xml

The image is published for linux/amd64 and linux/arm64. It runs as the non-root node user with WORKDIR /work and ENTRYPOINT ["node", "/app/dist/bin.js"], so any wirebench subcommand follows the image reference directly — mount the project under /work and pass secrets with -e.

npx

WIREBENCH_SECRET_BILLING_PASSWORD="$BILLING_PASSWORD" \
  npx --yes @wirebench/cli@5.0.0 run ./project --env staging --baseline --reporter junit=reports/wirebench.xml

Works on any runner that already has Node 24 — the same package the GitHub Action installs under the hood, with no other setup.

--baseline also compares each response with its committed golden; see Snapshot regression.

Mocks in a pipeline

wirebench mock serves a project’s mock services without the app, so the tests in a pipeline can call a mock instead of an upstream system that is down or does not exist yet. It prints a listening <name> <url> line per mock, then one line per request (--json for one JSON object per line), and serves until it gets SIGINT or SIGTERM, which exits 0.

npx --yes @wirebench/cli@5.0.0 mock ./project orders --port 8089 > mock.log &
MOCK=$!
until grep -q '^listening ' mock.log; do sleep 0.2; done
npx --yes @wirebench/cli@5.0.0 run ./api-tests --env mocked
kill -TERM "$MOCK"; wait "$MOCK"

In the container image the mock listens on every interface (WIREBENCH_MOCK_HOST=0.0.0.0), since loopback inside a container is out of reach; publish the port and docker stop ends it cleanly:

docker run -d --name orders-mock -p 8089:8089 -v "$PWD:/work" \
  ghcr.io/wirebench/wirebench-cli:5.0.0 mock ./project orders --port 8089

Off loopback, anyone who can reach the port can call it

Outside a container the mock listens on 127.0.0.1 unless you pass --host or set WIREBENCH_MOCK_HOST, and the project can never choose the address. A mock holds no secrets, but it answers whoever reaches it.

Name mocks by name, folder slug or id; with none, every mock in the project starts. A port that is taken, a definition that is not cached, or an unknown mock fails at once (exit 3 or 2) rather than leaving the tests to time out. See the CLI reference for every flag.

Before serving, wirebench mock check ./project [mock…] checks every stub against the contract, the way a received response is checked, and exits 1 when one does not conform, so a stub that drifted fails the job instead of a test passing against the mock that would fail against the real service. See wirebench mock check.

Secret tokens

A ${secret:name} token in a request reads WIREBENCH_SECRET_<NAME>, with the name upper-cased: ${secret:billing_key} reads WIREBENCH_SECRET_BILLING_KEY first. A token the workspace maps to a secret manager can also come from there (see Secrets from a secret manager); with neither, the request fails. A token reached through a property counts too. wirebench secrets list names the variable for each token the selected requests use, with secret "billing_key" as its purpose and whether it’s set:

npx --yes @wirebench/cli@5.0.0 secrets list ./project --env staging

It takes --var like run does. Pass it the same ones: a token that only a --var property holds is listed only when that --var is given.

A token that nothing supplies — no variable set, and no secret manager mapping that would provide it — fails that request with secret-missing — Set WIREBENCH_SECRET_BILLING_KEY to run “…” — rather than sending an empty value. See Secret tokens and scanning for how a value ends up behind a token.

Secrets from a secret manager

If the workspace maps a secret to a manager (see Secrets from external managers), wirebench run can fetch it with the manager’s CLI on the runner, using the runner’s own login. A WIREBENCH_SECRET_<NAME> variable still wins.

The runner must trust the mapping, because the mapping decides what a run can read. Pin the one you reviewed:

wirebench secrets list ./projects/orders
wirebench run ./projects/orders -e ci --trust-secret-sources-hash <hash printed above>

A pushed change to the mapping then fails the run until the pin is updated. --trust-secret-sources trusts whatever is there, and --no-secret-sources turns sources off. The two trust flags can’t be combined. See Secrets from external managers for the kinds and the tool each needs on the runner.

secrets list shows where each secret would come from in a SOURCE column:

• env when the variable is set, since it always wins;
• the mapping’s kind (vault, aws, …) when a source would supply the value;
• kind (untrusted), such as vault (untrusted), when the run would refuse the mapping until it is trusted;
• invalid for an entry that doesn’t parse, and — when nothing supplies the value.

It also prints Secret sources hash: <hash>, followed by (trusted) when the flags you passed would trust it or (pass --trust-secret-sources-hash to trust it) when they would not. The hash is the value --trust-secret-sources-hash takes. A name only a source supplies is listed as mapped and counts as supplied for the exit code; the run itself refuses an untrusted mapping and prints the hash.

wirebench send, wirebench call and wirebench mcp take the same three flags. A value fetched in one MCP call stays masked in the calls after it, since the server keeps it for its whole life.

Callback assertions

A step or request with a callback assertion waits for a webhook at a catch URL of a Wirebench Server workspace. To check it in a pipeline:

• Set WIREBENCH_SERVER_URL to the server, and WIREBENCH_SERVER_TOKEN to a CI token of the workspace, stored as a pipeline secret. An editor or admin creates the token in Preferences → Devices & tokens. A blank value counts as unset.
• The token never appears in the output or the reports.
• A wrong or revoked token errors every callback assertion with the server’s answer, and the run exits 3. With the variables unset, callback assertions error the same way and the rest of the run goes on.

Contract changes as a gate

wirebench diff-contract <old> <new> compares two versions of a WSDL or of an OpenAPI document and fails the job when a change would break an existing client: a removed operation or field, a field made required, a narrowed type or enumeration, a moved endpoint. Either side can be a file, a URL, or project:<name> — the definition the project already caches for that interface or API — so a job can check the contract it just built against the one the project was built from:

npx @wirebench/cli diff-contract project:Orders build/openapi.yaml --reporter markdown=contract-diff.md

Exit 0 when nothing breaks and 1 when something does; --fail-on any fails on any change, and --fail-on none only reports. --reporter markdown=<file>, html=<file> and json=<file> write the report, breaking changes first. The CLI reference lists what counts as breaking.

What comes back

Every recipe above is the same CLI underneath, so the outcome is identical whichever one runs it:

• Exit 0 when every selected request passed, 1 when an assertion failed, 2/3 for a pipeline problem (bad flags, an unresolved secret, a network failure) — see the CLI reference for the full exit-code table.
• --baseline adds a comparison with each request’s committed golden: a difference exits 1, and a golden that cannot be read or is too large (over 2 MiB) to compare exits 3. Under --require-baseline, a request with no golden exits 3 too.
• --reporter junit=<file> writes a JUnit file naming the request and the assertion that failed, for the CI system’s own test view.
• Nothing prints a secret’s value: it’s masked wherever it could appear — headers, URLs, bodies, an assertion’s actual text — in every reporter, including the plain cli one.

Related

• Project format — the mocks/ folder wirebench mock serves.
• Secrets — how a secret reference resolves, on the desktop and in the CLI.
• CLI reference — the full wirebench run and wirebench secrets list reference, including every assertion type and report format.