Can an AI agent read your docs and act on them?

Our agent starts every drive at the docs. Most were good enough to act on; llms.txt and OpenAPI helped it most.

What we found.

Docs that hold up for an AI agent, by part. An index that lists the docs: 78 of 143, One topic per page, under headings: 141 of 143, Code on API reference pages: 84 of 123, A quickstart with a first request: 100 of 143.

Share this card: Wide PNG · Tall PNG

How an agent reaches the API docs. Docs linked on the home page: 101 of 119, llms.txt lists the docs: 93 of 119, An API reference page: 100 of 119, llms.txt lists the API reference: 80 of 119, No link and no llms.txt: 8 of 119.

Share this card: Wide PNG · Tall PNG

Show all 5Show fewer
What SaaS robots.txt files tell AI. A Content-Signal line: 42 of 200, Says ai-train=no: 12 of 200, Says ai-input=no: 0 of 200, A Content-Usage line: 0 of 200.

Share this card: Wide PNG · Tall PNG

1,428: AI crawlers fetched our Markdown pages in one week, and our llms.txt 7 times.

Share this card: Wide PNG · Tall PNG

15%, 30 of 204: 30 of 204 products we drive publish an llms.txt that meets the bar for agents.

Share this card: Wide PNG · Tall PNG

Everything on Docs and llms.txt.

Research and data 5

  1. Can an AI agent find its way through your docs?44 of 143 products' docs give an AI agent all four things it needs to find its way. The pages themselves hold up best. The code samples and the index are where docs fall short.
Show all 5Show fewer
  1. Can an AI agent reach the API docs from a product's home page?101 of 119 products (85%) link their docs in the home page's HTML, one link from home for an AI agent. At 8 of 119, an agent that follows links finds no way to the docs: no link on the home page and no llms.txt that lists them.
  2. Content Signals and AI rules in SaaS robots.txt42 of the 200 SaaS products we checked (21%) tell AI what it may do with their content in robots.txt, all with Cloudflare's Content-Signal line. None uses the IETF's Content-Usage line.
  3. Do AI agents read llms.txt and Markdown?AI crawlers fetched our Markdown pages 1,428 times in one week and our llms.txt 7 times. AI assistants working for a person asked for Markdown 15 times, and for llms.txt not once.
  4. Does your llms.txt help an AI agent?168 of 204 products we drive publish an llms.txt, and 30 of them meet the bar: a title, a summary, sections of working links, under 50,000 characters.

Protocols and specs 3

  1. llms.txtA Markdown map of a product's docs, written for language models.168 of 201 products we drove offer it
Show all 3Show fewer
  1. OpenAPIA machine-readable description of every route of an API.103 of 201 products we drove offer it
  2. robots.txt for AI agentsThe file that tells crawlers and agents which pages to leave alone.

Fix guides 1

  1. Docs AI agents can read: llms.txt, Markdown, OpenAPIAn llms.txt, Markdown twins of your pages, an OpenAPI spec and an MCP server card.33 of 201 products without an llms.txt

The products, pass and stop.

Docs our agent could act on. 145

Show all 145Show fewer

Docs it could act on only in part. 50

Show all 50Show fewer

See where an agent stops in yours.

Free: what an agent finds on your site, any wall in its way and how many things to fix.