What we found.
What the errors our agent met on real calls said
| Of 62 products | Products | Share |
|---|---|---|
| Every error said what was wrong | 19 of 62 | 31% |
| At least one said what was wrong | 55 of 62 | 89% |
| At least one gave a status without the cause | 39 of 62 | 63% |
| At least one gave nothing to act on | 18 of 62 | 29% |
Whether the docs describe the API's errors
| Of 141 APIs | APIs | Share |
|---|---|---|
| Document their error format | 109 of 141 | 77% |
| Document them in part | 13 of 141 | 9% |
| Don't document them | 19 of 141 | 13% |
What the APIs answered four bad requests with
| Of 35 APIs | APIs | Share |
|---|---|---|
| Named what was wrong every time | 11 of 35 | 31% |
| Named it at least once | 33 of 35 | 94% |
| Answered every error in JSON | 21 of 35 | 60% |
| Answered with a web page | 8 of 35 | 23% |
| Used Problem Details (RFC 9457) | 0 of 35 | 0% |
| Linked to their docs | 7 of 35 | 20% |
| Gave a request id | 16 of 35 | 46% |
Whether the APIs said their rate limit
| Of 35 APIs | APIs | Share |
|---|---|---|
| Sent a rate-limit header | 5 of 35 | 14% |
| Said how many calls are left | 5 of 35 | 14% |
| X-RateLimit-* headers | under 5 | – |
| IETF RateLimit fields | under 5 | – |
- 33 of the 35 named what was wrong at least once: the missing key, the unknown route or the broken body. The rest only repeated the status, like "Unauthorized" or "Not found".
- 21 answered every bad request in JSON, which an agent can read without guessing. 8 answered at least one with a web page instead.
- None of them used Problem Details (RFC 9457, application/problem+json).
- 7 linked to their docs in an error, so an agent can look up the fix.
- 16 gave a request id, which support can trace.
- 5 of the 35 sent a rate-limit header, so most leave an agent to find its limit at the first 429.
- None of them answered a route that doesn't exist or a broken body with success.
What to do.
For product owners
- Say what was wrong and how to fix it: the missing key, the field, the route. A status alone tells an agent nothing new.
- Answer every error in JSON, also at your edge and for routes that don't exist.
- Use Problem Details (RFC 9457) and put a docs link in each error.
- Send RateLimit headers on every answer, so an agent can slow down before a 429.
For agent builders
- Read the error body, not just the status. 33 of the 35 APIs here named what was wrong at least once.
- Check the content type before you parse an error. Some answers are web pages, not JSON.
- Don't count on rate-limit headers: most APIs here sent none. Back off on a 429 and honour Retry-After when it comes.
Method and limits.
How we tested
- Errors on real calls: 62 products where our agent met at least one error while it worked the API, MCP server or CLI, on drives from 1 Oct 2026 to 1 Nov 2026. An error is clear when it says what was wrong, vague when it gives a status without the cause, and silent when it gives nothing an agent can use.
- Error docs: on drives from 1 Oct 2026 to 1 Nov 2026, our agent read the docs of 141 products with a public API and noted whether they document the errors the API returns.
- On 25 Oct 2026 we sent four fixed bad requests to 40 APIs of products we drove: no key, a made-up key, a route that can't exist and a broken JSON body. 35 answered.
- Each API is the host and route our agent used on its drive.
- A message counts when it names what to fix (the key, the route or the body), not when it only repeats the status.
- Rate-limit headers: the IETF RateLimit fields, X-RateLimit-* and Retry-After.
Limits
- Errors on real calls come from the work each drive did, so the calls differ from product to product.
- One day's answers, four requests per API, no valid key. Errors after a sign-in may read differently.
- The test makes no successful call, so headers an API sends only on success aren't counted.
- Counts under 5 are left out.
See what an agent sees in yours.
Free: what an agent finds, any wall in its way and how many things to fix. The Audit ($499) gives you every fix.