> ## Documentation Index > Fetch the complete documentation index at: https://docs.armature.tech/llms.txt > Use this file to discover all available pages before exploring further. # Scanner methodology > What the Agent Discoverability scanner checks, why each check matters, and how results work The Agent Discoverability scanner tests one public domain. It checks whether agent systems can reach, identify, and find the product's official public surfaces. The scan is a static audit. It does not run Claude, Codex, or another live agent. It does not measure whether an agent selects the product during a task. ## How the scan works 1. We normalize the submitted domain and require public HTTPS. 2. We read the homepage, robots rules, sitemap, selected documentation pages, and public discovery files. 3. We check supported public registries and directories. 4. We run two Exa searches. One searches for the brand. One searches for official developer resources. 5. We evaluate the same 45 deterministic tests against the collected evidence. We do not assign a score or grade. The report shows the full result mix and the actions that can improve discoverability. ## Result meanings | Result | Meaning | | ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------- | | **Pass** | The scan found the expected public evidence. | | **Warning** | The surface exists, but the evidence is incomplete or weak. | | **Fail** | The expected public evidence is missing or invalid. | | **Not applicable** | The product does not advertise the related surface. For example, MCP checks do not apply when the product does not advertise an MCP server. | | **Not tested** | The scan did not collect enough evidence for a safe result. | | **Error** | The scanner could not finish the test. This is a scanner-side result, not a failure by the website. | ## Agent access and indexability These 22 tests check whether agent systems can reach and read official public content. | Test | What we verify | Why it matters | | ------------------------------------- | ---------------------------------------------------------------------------------------- | ---------------------------------------------------------------------- | | Public homepage reachability | The HTTPS homepage returns useful public content. | Agents need a stable first entry point. | | Redirect outcome | Entry URLs resolve to one related canonical HTTPS host. | Agents must keep the correct product identity through redirects. | | Server-rendered content | The initial HTML contains the product name, description, and main heading. | Many crawlers do not run page JavaScript. | | Search-crawler robots policy | `robots.txt` permits supported search crawlers at public paths. | Search indexes cannot retrieve blocked content. | | User-triggered-agent robots policy | `robots.txt` permits `ChatGPT-User` and `Claude-User` at public paths. | User-requested agent visits need explicit access. | | Crawler user-agent reachability | Approved crawler user agents receive the same useful page as a normal browser. | Bot challenges and empty bot pages can hide the product. | | Sitemap presence | A sitemap exists at a standard path and is declared in `robots.txt` when available. | A sitemap gives crawlers a stable page inventory. | | Sitemap URL validity | Sampled sitemap URLs are public, safe, and reachable. | Broken or unrelated entries reduce discovery quality. | | Sitemap freshness | Sitemap entries use valid `lastmod` dates when dates are present. | Freshness data helps indexes select current pages. | | Documentation linked from the website | The homepage links to public docs with a clear label. | Agents need a direct route from product identity to product knowledge. | | First-party documentation domain | External docs are linked by the official homepage and keep the same product identity. | The homepage link proves that an external docs host is official. | | Documentation page sample | The scan can read linked overview, setup, and reference pages. | A single docs page is not enough for product use. | | Public documentation access | Sampled docs work without sign-in, CAPTCHA, consent wall, or bot challenge. | Public discovery fails when core knowledge needs a session. | | `llms.txt` presence | `/llms.txt` or `/.well-known/llms.txt` returns real text or Markdown. | The file gives agents a compact map of official resources. | | `llms.txt` structure | The file has a title, a short product description, and grouped official links. | Clear structure helps agents select the right source. | | `llms.txt` linked pages | Sampled links are public, valid, and tied to verified product surfaces. | A map is useful only when its destinations work. | | Agent instruction file | `/agents.md` explains purpose, supported tasks, limits, and official resources. | Agents need clear product-specific operating guidance. | | Advertised Markdown alternate | HTML or HTTP headers advertise a valid Markdown version. | Agents can choose a compact, machine-readable representation. | | Markdown content negotiation | `Accept: text/markdown` returns Markdown for the same page and sets the correct headers. | One URL can serve both browser and agent formats. | | HTTP discovery links | HTTP `Link` entries expose at least two useful first-party resources. | Protocol-level links help clients find machine-readable surfaces. | | Canonical page metadata | Canonical URL, title, description, and Open Graph data use one product identity. | Consistent identity helps search systems verify the official source. | | Retrieval size | Sampled pages stay within a bounded normalized token size. | Focused pages are easier and cheaper for agents to retrieve. | ## Agent-facing distribution surfaces These 21 tests check whether the product appears where agents and developers look for tools. Surface-specific tests become **Not applicable** when the product does not advertise that surface. | Test | What we verify | Why it matters | | --------------------------- | -------------------------------------------------------------------------------------------- | ------------------------------------------------------------ | | ARD catalog | `/.well-known/ai-catalog.json` is valid and lists public first-party resources. | A standard catalog gives agents one discovery endpoint. | | ARD entry validity | Catalog items have stable IDs, names, media types, and working content URLs. | Valid entries let clients use the catalog without guessing. | | Agent Skills index | The standard index links to useful public skill artifacts. | Skills can tell agents how to use the product. | | A2A agent card | The standard agent card has a consistent product identity and useful capabilities. | A2A clients need verified metadata before interaction. | | Public OpenAPI document | A valid OpenAPI 3 document is public at a standard or linked URL. | Agents can discover and understand the public API. | | API catalog | The RFC 9727 catalog links to public first-party API descriptions. | A catalog lets clients find several official APIs. | | Public MCP advertisement | An advertised MCP server has a public endpoint and identity tied to the domain. | Agents need proof that an MCP endpoint is official. | | Official MCP Registry | The advertised MCP server has a registry entry tied to the domain or verified repository. | The registry is a common MCP discovery channel. | | OpenAI Apps directory | An advertised OpenAI app has an official directory result with verified product links. | Directory presence helps ChatGPT users find the integration. | | Claude Connectors directory | An advertised Claude connector has an official directory result with verified product links. | Directory presence helps Claude users find the integration. | | Vercel Marketplace | A matching marketplace result links to official product surfaces. | The marketplace is a developer discovery channel. | | Context7 | A matching Context7 project links to the official domain or repository. | Context7 can surface product documentation to coding agents. | | Smithery | An advertised MCP server has a matching, identity-verified Smithery result. | Smithery is an MCP discovery channel. | | mcp.so | An advertised MCP server has a matching, identity-verified mcp.so result. | mcp.so is an MCP discovery channel. | | Glama | An advertised MCP server has a matching, identity-verified Glama result. | Glama is an MCP discovery channel. | | PulseMCP | An advertised MCP server has a matching, identity-verified PulseMCP result. | PulseMCP is an MCP discovery channel. | | skills.sh | A matching skill links to the verified repository and domain. | Skill directories can distribute agent instructions. | | Official GitHub repository | The homepage or docs link to a public repository with matching identity. | Repositories are a major source for coding agents. | | Repository agent files | The verified repository includes relevant `AGENTS.md`, `SKILL.md`, or discovery links. | Agent files expose guidance where coding agents work. | | Official packages | Advertised packages link back to verified official surfaces. | Package registries are a common developer discovery path. | | Developer portal | A public developer entry page links to setup, reference, examples, updates, and support. | One clear entry page reduces search and navigation work. | ## Agent-oriented search presence These two tests use Exa as an agent-oriented search index. Each search stores at most 10 result rows with URL, title, and an identity verdict. | Test | What we verify | Why it matters | | ------------------------- | ------------------------------------------------------------------------------------------ | --------------------------------------------------------------------- | | Brand search | A search for the exact brand returns an official first-party surface near the top. | Agents often start with a product or company name. | | Developer-resource search | A search for the brand with `developer docs API MCP` returns official developer resources. | Agents need to find implementation content, not only marketing pages. | The scan does not generate category queries, problem queries, comparison queries, or agent selection experiments. Those are separate Agent Discoverability runs. ## Recommendations Every warning and failure includes a direct action. The report ranks the most important actions first. It also keeps the complete test table so you can inspect the result, evidence, and action for each test. ## Cache and public results A completed result stays fresh for exactly 24 hours. A repeat scan during that period returns the same result and starts no new work. An expired result stays idle until a user submits the domain again. Completed results can appear in the public recent-results list. The scanner does not publish failed runs, internal IDs, request identifiers, raw provider records, or scanner secrets. ## Safety limits The scanner makes read-only requests to public HTTPS resources. It blocks private, local, link-local, metadata, and other prohibited IP ranges. It rechecks DNS and every redirect to prevent rebinding. Requests have strict limits for time, redirects, response bytes, decompression, and total request count. The scanner never sends target credentials. It does not create accounts, make purchases, or call state-changing MCP tools.