request_capability, records an unmet-capability signal and changes nothing else. Workspaces with Evaluation enabled see six more tools, listed under Evaluation tools below. Two of them start eval runs, and they are the only tools here that change anything.
list_mcp_servers
Your registered production MCP servers, with connection metadata. No secrets are ever returned.
list_sessions
Reconstructed agent sessions, newest first — each with its primary use case and latest judged outcome. Mirrors the Sessions page.
get_session
One session end-to-end: the intent, every call in order, classification, and the judge’s verdict. This is session replay as data.
list_use_cases
Use-case clusters with volume, pass rate, and share of traffic. Mirrors Use cases. On young workspaces it falls back to raw-intent grouping and says so.
list_issues
Issues ranked by how many sessions they affect, with session KPIs, daily pass-rate buckets, cause category, and affected use cases. Mirrors Issues.
get_issue
One issue in full: the root cause, the sessions, tools, use cases, and clients it affects in the range, and example sessions to replay.
request_capability
Agents call it when the user needs something no listed tool can do. The call is acknowledged and recorded as unmet demand — it’s the same tool the analytics SDK injects into instrumented servers.
Evaluation tools
These six tools appear on workspaces with Evaluation enabled. They cover the full loop an agent needs: find the suite, run it, and read the verdict. The tworun_* tools require the editor, admin, or owner role.
list_eval_suites
Your active eval suites, each with the MCP server it tests and the number of eval cases it would actually run. Start here to get a testSuiteId. The case count reflects what Run suite dispatches, so archived cases are already excluded.
list_eval_cases
The eval cases in one suite, exactly the set run_eval_suite dispatches. Each case carries its criteria count, schedule type, the other suites it belongs to, its last outcome, and a pass rate over recent runs. Runs the judge could not score are excluded from that rate, never counted as failures.
get_eval_case
One eval case in full: what the tester is instructed to do, the criteria it is judged against, its tool policy and limits, the suites it belongs to, and its recent individual runs with outcome, score, duration, and any error. Use it to understand why a case failed before changing anything.
run_eval_case
Starts one individual run of one case, on one target. When the case has exactly one active target, that target runs; when it has several, the case’s default model runs. To cover every configured target, run the case from the dashboard or pass testerTargets to the CI endpoint. The tool returns the run id immediately and the run executes asynchronously, so poll get_eval_case for the outcome.
run_eval_suite
Starts one suite run over every case in a suite, the same action as Run suite in the dashboard. Membership is snapshotted at dispatch. The tool returns the suite run id and how many cases were queued; poll get_suite_run for the result.
get_suite_run
One suite run: overall status, how many cases completed and passed, whether the result still matches the suite’s current membership, and the per-case outcomes with each underlying run. Omit suiteRunId to read the latest suite run, which answers “is this MCP server ready to ship right now”.
Prompts
triage_issues — “Rank this workspace’s issues by blast radius, open the worst one, and show a session that hits it.” A guided tour of list_issues → get_issue → get_session.