The report your team gets.

Where agents stop in your product, what to fix first and the level each fix gets you. Here for Quillbill, a made-up invoicing tool.

Quillbillquillbill.example · InvoicingExample · fictional product

Agent levelLevel 1 · Needs a personAgents can't get in on their own.

L1L3after fixes 1 and 2

Driven: 24 Sep 2026
Drive: 47 min
Evidence: 40 attempts

Stops at sign up: a bot check on the sign-up form.
Two fixes to L3: skip the CAPTCHA for signed agents, hand out a key over the API.
7 of 8 core features already run once an agent is in.

Get a Drive · $49 Compare plans

Report in 24 hours
Or your money back

Where agents stop, and why.

Our agent stopped at sign up: a bot check on the sign-up form.

Sign up: a bot check on the sign-up form (Cloudflare Turnstile).
What came back: POST /signup, answered 403 turnstile_required.
Who it stops: every agent, on any network. A person has to do this step.

Who gets in

Agents on a serverStopped by a CAPTCHA on the sign-up form.
Signed agentsThe sign-up page opens. The CAPTCHA on the form stops them too.
Agents in a person's browserNot measured yet.

Sign-up page checked 24 Sep 2026. Signed agents: the ones Cloudflare verifies. Where agents come from

Every way in, as our agent tried them (2)

Ways in
Way in	What happened
Email and password	Stopped by a bot check
Continue with Google	Not tried: Google blocks automated sign-in

What the agent sent, and what came back

Quillbill's sign-up form is protected by Cloudflare Turnstile. Without a token that only a person can produce, the server answers 403. The agent tried the documented form, the same form in a real browser and the API docs for another way in. There is none, so every agent stops here.

What the agent sent

POST https://app.quillbill.example/signup
content-type: application/json
signature-agent: "https://agent.agentdrives.com"

{"email":"drive-142@agentdrives.com","password":"••••••••","company":"AgentDrives test"}

What came back

HTTP/2 403
content-type: application/json

{"error":"turnstile_required","message":"Please complete the security check."}

Fix these first.

In the order they move your level: L1 today, L3 after fixes 1 and 2.

L1Today
L3After fixes 1–2
L4After fixes 1–4
L5After fixes 1–5

1
Skip the CAPTCHA for signed agents
Opens the door to every signed agent. The biggest single fix.
- Sign up and verify
- Engineering, a sprint
How
Skip Turnstile for requests with a valid Web Bot Auth signature from a known agent (Cloudflare's verified bots list), or add POST /v1/signup that sends a one-time code to the email address. Keep rate limits per domain.
Step-by-step guide
2
Hand out a key over the API
Without it an agent that signed up still needs a person.
- With fix 1: L3
- Key
- Engineering, days
How
After email verification, return a scoped API key from POST /v1/keys, or support the OAuth device flow so an agent can ask its human for approval once.
Step-by-step guide
3
Document the currency field
Every agent's first invoice fails on it.
- First action on the first try
- Docs, days
How
Mark currency as required in the OpenAPI spec for POST /v1/invoices, or default it to the organisation's currency.

2 more fixes

4
Let agents cancel and delete over the API
Agents, and the people they act for, need a way out.
- With fixes 1–3: L4
- Cancel and delete
- Engineering, days
How
POST /v1/subscription/cancel and DELETE /v1/organization with a confirmation token; send a confirmation email and keep the 30-day grace period you already have.
5
Take agent payments (x402 or MPP)
Lets agents pay you on their own, with no checkout page.
- With fixes 1–4: L5
- Pay
- Product, a quarter
How
Answer an unpaid API call or a plan purchase with HTTP 402 and an x402 or MPP challenge on your own domain, so an agent pays in the same request.
Step-by-step guide

What agents can already do.

7 of 8 core features ran, each result read back.

Works

Learn the whole product from the docs, llms.txt and OpenAPI spec
Create, update and archive clients
Create invoices with line items, taxes and due dates

5 more

Send an invoice by email and read its status back
Record a payment and mark the invoice paid
Schedule payment reminders
Export invoices as PDF and CSV
Subscribe to webhooks for paid and overdue invoices

Not yet

Create an accountA bot check on the sign-up form only a person can pass
Get an API keyKeys can only be made in the dashboard
Set up recurring invoicesRecurring schedules exist in the dashboard, not in the API

3 more

Connect a bank accountThe bank connection runs through a provider's browser flow
Upgrade or pay for a planCard only, in the hosted checkout
Delete the accountOnly by emailing support

What an agent finds: API, OpenAPI spec (One required field missing), MCP server, llms.txt, Webhooks, SDK. Not found: CLI, Sandbox.

Your docs, read and judged: all 186 pages read. Some gaps.

A getting-started page.
An auth page.
Errors only mentioned in passing.
Code examples on most pages.
Newest page updated 18 Sep 2026.
A page for 7 of 8 core tasks.

Core tasks with no page of their own: Connect a bank account.

Tried as documented: 11 of 12 documented ways to do the core work worked when tried.

1 documented way didn't work.
1 thing worked that the docs don't describe.

Documented, and didn't work:

Send an invoice, through the MCP server (send_invoice): the tool needs a recipient its description leaves out

Safe when agents get it wrong.

5 mistakes an agent makes, and what Quillbill answered.

Clear
Create an invoice with a malformed body400 naming the field (lines[0].amount) and the type it expects
Clear
Call with a wrong key of its own401 invalid_api_key, with a link to the docs on keys
Silent
Send the same create request twice, as a retry wouldTwo invoices created, no warning; no Idempotency-Key support

2 more

Vague
Read a client that does not exist404 with an empty body: no error code, nothing to act on
Clear
A burst of 15 reads to one endpoint429 on the 13th, with Retry-After: 10; the agent stopped there

What it means

Most errors say what to fix and the rate limit is honest, with a Retry-After. The weak spot is retries: sending the same invoice twice creates two, and there is no Idempotency-Key, so an agent that retries after a timeout bills a client twice. Run three times, 6 of the 7 core features worked every time; sending an invoice failed once with a 502.

3 of 5 answers say what to fix.
One mistake without an error: two invoices created, no warning; no Idempotency-Key support.
Run three times, 86% of the core work held up every time.

Paying and leaving.

Today an agent needs a person to cancel and delete its account.

PayReaches the checkoutReaches the checkout: stopped at the card form. Upgrading needs a card in the hosted checkout, and the trial asks for one too. There is no billing API and no agent payment rail.Fix 5: Take agent payments (x402 or MPP)
CancelNeeds a personCancelling is only in the dashboard (Billing › Plan), behind a two-step retention screen.Fix 4: Let agents cancel and delete over the API
DeleteNeeds a personNo delete option in the API or the dashboard. The help centre says to email support.Fix 4: Let agents cancel and delete over the API

Your checkout, walked to the last click

Reached the card form and stopped: Pro, monthly.
Checkout: Stripe Checkout.
An invisible bot check loaded at checkout.
Wallets: Link.

Audit

Every documented route, checked.

131 of the 160 documented routes and tools ran on a test account; 120 do exactly what the docs say.

Ran: 131 of 160
Do what the docs say: 120
Don't: 11

Fix first

Document the error responses of every operation68 operations list only the 200 response; an agent can't tell a missing field from a bad key without trying.
Name the required arguments of each MCP tool7 of 18 tool descriptions leave out an argument the server rejects without.
Fix the invoice-list filtersstatus=overdue is documented, the API answers 400 unknown_filter.

The operations behind each gap

Document the error responses of every operation: op-014, op-022, op-031
Name the required arguments of each MCP tool: mcp-003, mcp-009
Fix the invoice-list filters: op-041

Docs pages to correct (2)

docs.quillbill.example/api/invoices/list Documents a status=overdue filter the API rejects
docs.quillbill.example/guides/pagination Shows page/per_page; the API uses a cursor since v2

Not in your docs (1)

GET /internal/v2/recurring Seen in the dashboard's own calls.

What ran

The API reference is complete for most operations and matches what the API does for 120 of the 131 we ran. The gaps cluster in two places: error responses, which are documented for fewer than half of the operations, and the MCP tools, whose descriptions leave out required arguments. Two pages describe parameters the API no longer accepts. One dashboard endpoint works with an API key but is not in the docs.

Ran 131 of the 131 items that are safe to run. 29 left out because they spend money or reach real people.

Audit

Your rivals, side by side.

2 of 3 rivals get an agent further than Quillbill. After fixes 1 and 2, you're level with the best.

Quillbill and its rivals: the level and where an agent stops
Product	Level	Where an agent stops
Ledgerkite	L3	Got in; a wall at pay
Sumsprout	L2	Stopped at key
QuillbillYou	L1	Stopped at sign up
Paperwren	L1	Stopped at sign up

Audit

Only your team sees this.

3 notes from the security review, each with its fix.

Medium
API keys never expire
Keys made in the dashboard have no expiry and no last-used date. A leaked key from an agent's environment stays valid until someone notices. Add an optional expiry and show last use.
Medium
Webhook secret shown in full after creation
Settings › Webhooks shows the signing secret in plain text every time the page loads, so anyone with dashboard access can read it. Show it once, then mask it.
Low
Sign-up reveals existing accounts
POST /signup answers "email already registered" for known addresses, which lets anyone check who uses Quillbill. Return the same answer for both and send an email instead.

Routes outside your docs (1)

GET /internal/v2/recurring: the dashboard's recurring-invoice endpoint works with an API key (undocumented, so not counted in the level)

Not tested, and why (2)

Bank connection: runs through the provider's browser flow with a real bank login
Multi-currency payouts: needs a verified business account

Your level, where agents look.

Listed in our directory with your level, and a dated badge for your site.

Owner-paid drive

Quillbillquillbill.example · InvoicingExample · fictional

Agent levelLevel 1 · Needs a personAgents can't get in on their own.

Holds up: 70 of 100, fair
Docs match: 92 of 100, good

Security: reported privately to the owner

The badge on your site

A badge for your site with your level and the date of the drive.
In the directory on the web and over our MCP server, where agents look software up.
Your reply next to the result, with its date.

Prove the fix.

Once a fix is live, a new drive gives you the new level and a new report.

Drive again, $49Any time after a fix: a new level and a new report within 24 hours.
Monthly drive, $29 a monthOur agent drives it every month. You hear when your level or the place an agent stops changes.
Your second drive, includedPart of an Audit: within 30 days of the report, to prove the fix.

Every step, with its proof.

40 attempts on 24 Sep 2026, in 47 minutes, each with what the agent read back.

Each checkpoint, with what happened

1
DocsHeadless
Read /llms.txt, the OpenAPI 3.1 spec (142 operations) and the Markdown docs. Learned clients, invoices, payments, reminders and webhooks in 4 minutes.
2
Sign upBlocked
POST /signup returns 403 without a Turnstile token. In a real browser the check appears as an interactive challenge. We never solve CAPTCHAs, so the agent stopped here.
3
VerifyNeeds a human
Not reached on its own. The owner's test account was already verified.
4
KeyNeeds a humanowner's test account
Keys are made under Settings › API in the dashboard only; there is no API or OAuth route to get one. The owner created a key for the drive.
5
First callHeadless
GET /v1/me returned the test organisation with the owner's key. MCP initialize and tools/list worked too (18 tools).
6
First actionHeadless
Created a client and a draft invoice with two lines, then read both back. The first try failed on an undocumented required field (currency).
7
PayNeeds a human
Upgrading needs a card in the hosted checkout, and the trial asks for one too. There is no billing API and no agent payment rail.
8
CancelNeeds a human
Cancelling is only in the dashboard (Billing › Plan), behind a two-step retention screen.
9
DeleteNeeds a human
No delete option in the API or the dashboard. The help centre says to email support.

All 40 attempts

Time	What the agent tried	Route	Result	What it read back
00:00	Read and judge the docs (our side, before the agent)	docs pass	Done	186 of 186 pages: errors only mentioned in passing, no page for one core task
00:02	Read llms.txt and the docs index	GET /llms.txt	Done	200, 48 links to Markdown pages
00:03	Load the OpenAPI spec	GET /openapi.json	Done	OpenAPI 3.1, 142 operations
00:05	Find a sign-up route for agents	docs search	Failed	No sign-up endpoint or agent route documented
00:06	Sign up with the form	POST /signup	Blocked	403 turnstile_required
00:08	Sign up in a real browser	browser /signup	Blocked	Interactive Turnstile challenge shown; not solved
00:09	Continue with Google, our agent's own account	browser, Google sign-in	Skipped	Google: this browser may not be secure; not tried
00:10	Use the owner's test key	owner-provided	Done	Key handed over for the drive
00:11	Identify the organisation	GET /v1/me	Done	200, org "AgentDrives test"
00:12	Connect the MCP server	MCP initialize + tools/list	Done	18 tools listed
00:13	Create a draft invoice	POST /v1/invoices	Failed	422 currency is required (not in the spec)
00:14	Create a draft invoice with currency	POST /v1/invoices	Done	201, read back with GET /v1/invoices/inv_7Hq2
00:15	Create a client	POST /v1/clients	Done	201, read back
00:16	Update and archive a client	PATCH /v1/clients/{id}	Done	archived=true on read-back
00:18	Add tax and a due date	PATCH /v1/invoices/{id}	Done	total incl. 21% VAT on read-back
00:19	Send the invoice to our test inbox	POST /v1/invoices/{id}/send	Done	Mail received in drive-142 inbox; status=sent
00:21	Record a payment	POST /v1/payments	Done	Invoice status=paid on read-back
00:22	Schedule a reminder	POST /v1/invoices/{id}/reminders	Done	Reminder listed for +7 days
00:23	Export as PDF	GET /v1/invoices/{id}.pdf	Done	application/pdf, 2 pages
00:24	Export as CSV	GET /v1/invoices?format=csv	Done	CSV with 1 row
00:25	Subscribe to invoice.paid	POST /v1/webhooks	Done	Test event delivered, signature valid
00:26	Create a recurring invoice	API docs	Failed	No recurring endpoint in the API
00:27	Dashboard route for recurring invoices	GET /internal/v2/recurring	Done	200 with API key; undocumented, kept out of the level
00:28	Connect a bank account	dashboard	Skipped	Provider browser flow with a real bank login; out of scope
00:30	Create an invoice over MCP	MCP create_invoice	Done	Invoice inv_8Kd1 read back over the API
00:31	Claims: send an invoice over MCP	MCP send_invoice	Failed	invalid_params: recipient is required (not in the tool's description)
00:31	Upgrade to Pro	GET /v1/billing	Failed	404; checkout is browser-only
00:33	Look for an agent payment rail	402 probe, /.well-known	Failed	No x402/MPP, no machine-readable pricing
00:36	Walk the hosted checkout to the last click	browser /billing/upgrade	Partly	Stripe Checkout, Pro monthly; stopped at the card form (no card on this drive)
00:38	Cancel the plan	browser /billing/plan	Needs a person	Two-step retention flow, dashboard only
00:40	Delete the account over the API	DELETE /v1/organization	Failed	404 not found
00:41	Find account deletion	help centre	Needs a person	"Email support to delete your account"
00:42	Crash test: create an invoice with a malformed body	POST /v1/invoices	Done	400, names lines[0].amount and the expected type
00:42	Crash test: call with a wrong key of its own	GET /v1/me	Done	401 invalid_api_key, with a docs link
00:43	Crash test: read a client that does not exist	GET /v1/clients/cl_none	Done	404, empty body
00:43	Crash test: send the same create request twice	POST /v1/invoices ×2	Done	Two invoices on read-back; no Idempotency-Key support
00:44	Check key lifetime	GET /v1/keys	Done	expires_at=null on every key
00:44	Crash test: a burst of 15 reads to one endpoint	GET /v1/invoices ×15	Done	429 on the 13th, Retry-After: 10; stopped at the first 429
00:45	Crash test: run the 7 core features twice more	API, as above	Done	6 of 7 worked on all three runs; send failed once with 502
00:46	Archive test data	DELETE /v1/invoices/{id}	Done	All test invoices voided

Details

Report AD-2026-0924-0142, a made-up product. Quote this number when you write to us.
Driven by the AgentDrives Agent on 24 Sep 2026, in 47 minutes, on a test account.
How the level is set

What you get with each.

A Drive gets you this report. An Audit adds the parts tagged Audit and a second drive within 30 days.

What's included	Drive$49a drive	Audit$299one time
A real agent signs up, does your core tasks, pays and leaves	Included	Included
Where it stops, and what to fix	With the evidence	With the evidence
Your level, a dated badge and your listing	With your level	Every part score
Every documented route, tool and command, against your docs	Not included	Included
Security review, and up to three rivals compared	Not included	Included
A second drive to prove the fix	Not included	Within 30 days
Ready	Report within 24 hours	Report within 24 hours
	Get a Drive	Order an Audit

See what an agent sees in yours.

A check shows what agents see. A drive proves what they can do.

The report your team gets.

Where agents stop, and why.

Fix these first.

What agents can already do.

Works

✕Not yet

Safe when agents get it wrong.

Paying and leaving.

Every documented route, checked.

Fix first

Your rivals, side by side.

Only your team sees this.

Your level, where agents look.

Prove the fix.

Every step, with its proof.

What you get with each.

See what an agent sees in yours.

Not yet