Skip to main content
Continuous integration (CI) is the pipeline that checks every change you push. Armature exposes an HTTP API so that pipeline can start an eval run and read the verdict, which lets you gate a deploy on your eval cases. The API triggers individual eval cases, one request per case. Suite runs start from the dashboard or from the run_eval_suite tool on the Armature MCP; there is no HTTP endpoint for them.

​
Authenticate

Mint an API key under Settings → API keys in the dashboard. Triggering runs requires a key from a member with the editor, admin, or owner role. Send the key as a bearer token on every request. US workspaces call https://app.armature.tech, and EU workspaces call https://eu.armature.tech.

​
Trigger a run

The API keeps the older route name workflows: a workflow id is an eval case id. Your agent can read case ids with list_eval_cases on the Armature MCP, and the dashboard puts the id in every case link (the drawer=workflow:<id> part of the URL). The endpoint answers 202 right away with the run’s id, and the run executes asynchronously. Runs started this way appear in the run history tagged ci. Useful body fields:

​
Read the verdict

Poll the run until its status is terminal, then read the evaluation:
The first call returns the run with its status and score. The second returns the judge’s verdict in full: every criterion mark with its evidence. Fail your pipeline step when the outcome is not Passed, or apply your own threshold from the score. One caution for strict gates: Passed means no applied criterion failed. The judge marks a criterion as not applicable when the run gave it nothing to check, and such a mark does not lower the score. When the gate must confirm every required behavior, check each criterion mark in the evaluation response instead of the outcome alone. Suites, runs, and scoring explains the rule.

​
Or let your agent do it

The same loop works through the Armature MCP without writing the HTTP calls yourself: run_eval_suite starts a suite run, and get_suite_run answers whether the suite is ready to ship. This fits a coding agent that just changed your MCP server and wants to verify the change before opening a pull request.

​
Limits

The trigger endpoint rate limits bursts, so trigger runs one by one rather than in a tight loop. Runs started from CI count toward the same included runs as every other run.