robots.txt for AI agents.
The file at a site's root that tells crawlers which pages to stay off. Some sites add rules that name AI agents.
Also called robots.txt.
In our drives.
Our agent reads a site's robots.txt before it opens a page and stays off a page the rules close to it. A sign-up page closed this way is shown as such on the product's page.
Products whose robots.txt asks crawlers to stay off the sign-up page.
14 of the 151 products we drove.
See where an agent gets stuck in your product.
Free: what an agent finds on your site and any wall in its way.