> ## Documentation Index
> Fetch the complete documentation index at: https://docs.armature.tech/llms.txt
> Use this file to discover all available pages before exploring further.

# Run evals from CI

> Trigger eval runs from your pipeline and read the verdict

Continuous integration (CI) is the pipeline that checks every change you push. Armature exposes an HTTP API so that pipeline can start an eval run and read the verdict, which lets you gate a deploy on your eval cases.

The API triggers individual eval cases, one request per case. Suite runs start from the dashboard or from the `run_eval_suite` tool on the [Armature MCP](/armature-mcp/tools#evaluation-tools); there is no HTTP endpoint for them.

## Authenticate

Mint an API key under **Settings → API keys** in the dashboard. Triggering runs requires a key from a member with the editor, admin, or owner role. Send the key as a bearer token on every request.

US workspaces call `https://app.armature.tech`, and EU workspaces call `https://eu.armature.tech`.

## Trigger a run

```bash theme={null}
curl -X POST "https://app.armature.tech/api/armature/v1/workflows/<EVAL_CASE_ID>/runs" \
  -H "Authorization: Bearer $ARMATURE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"reason": "release 2.3 gate"}'
```

The API keeps the older route name `workflows`: a workflow id is an eval case id. Your agent can read case ids with `list_eval_cases` on the [Armature MCP](/armature-mcp/tools#evaluation-tools), and the dashboard puts the id in every case link (the `drawer=workflow:<id>` part of the URL).

The endpoint answers `202` right away with the run's id, and the run executes asynchronously. Runs started this way appear in the run history tagged **ci**.

Useful body fields:

| Field            | What it does                                                                                                                                                                                                                                |
| ---------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `reason`         | The reason is shown next to the run in the dashboard, so give it your release or commit reference.                                                                                                                                          |
| `idempotencyKey` | The key makes a retried request return the same run instead of starting another one. It works when the request dispatches a single run.                                                                                                     |
| `testerTargets`  | A list of `{ "modelId": "<id>" }` entries, one run per entry. Each model must be one of the case's configured targets. Without this field, the case runs on one target: its single active target, or its default model when it has several. |

## Read the verdict

Poll the run until its status is terminal, then read the evaluation:

```bash theme={null}
curl "https://app.armature.tech/api/armature/v1/runs/<RUN_ID>" \
  -H "Authorization: Bearer $ARMATURE_API_KEY"

curl "https://app.armature.tech/api/armature/v1/runs/<RUN_ID>/evaluation" \
  -H "Authorization: Bearer $ARMATURE_API_KEY"
```

The first call returns the run with its status and score. The second returns the judge's verdict in full: every criterion mark with its evidence. Fail your pipeline step when the outcome is not **Passed**, or apply your own threshold from the score.

One caution for strict gates: **Passed** means no applied criterion failed. The judge marks a criterion as not applicable when the run gave it nothing to check, and such a mark does not lower the score. When the gate must confirm every required behavior, check each criterion mark in the evaluation response instead of the outcome alone. [Suites, runs, and scoring](/evaluation/suites-and-runs#the-judge) explains the rule.

## Or let your agent do it

The same loop works through the [Armature MCP](/armature-mcp/tools#evaluation-tools) without writing the HTTP calls yourself: `run_eval_suite` starts a suite run, and `get_suite_run` answers whether the suite is ready to ship. This fits a coding agent that just changed your MCP server and wants to verify the change before opening a pull request.

## Limits

The trigger endpoint rate limits bursts, so trigger runs one by one rather than in a tight loop. Runs started from CI count toward the same [included runs](/evaluation/suites-and-runs#included-runs) as every other run.