The Docket crawler
If Docket-SEO-Audit is in your logs, somebody audited your site with
Docket from their own computer. There is no Docket crawler farm and no scheduled sweep of the web:
every request under this name belongs to one audit that one person started. The one other thing
that sends requests from Docket is an occasional research survey, which has its own name and is
described below.
How it identifies itself
Every page request carries this user agent:
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0 Safari/537.36 Docket-SEO-Audit/1.0 (+https://docketseo.app/bot/)Pages that are also opened in a browser engine (to see what JavaScript builds) carry the same name and the same link back to this page.
One check borrows crawlers' names, and says so
Unless the audit runs offline, one check asks your server how it treats the crawlers that
decide whether you are found. It requests your homepage once under each of these user agents:
Googlebot, OAI-SearchBot, GPTBot, ChatGPT-User, PerplexityBot, Perplexity-User, ClaudeBot, Claude-SearchBot, Claude-User and Meta-WebIndexer. Each is that crawler's own published string, with this added to the
end so your logs show the request was Docket's:
Docket-SEO-Audit/1.0 (+https://docketseo.app/bot; owner-initiated access check)The crawler's name stays at the front because that is what a firewall or CDN rule matches.
Without it, the answer would be the one your server gives Docket rather than the one it gives
Googlebot, and the check exists to find servers that turn those crawlers away. These requests go
only to the site being audited. An offline audit (docket audit --offline, or
“Keep it private” in the app) sends none of them.
How hard it pushes
- At most 4 requests at a time, with at least 0.25 seconds between requests to your site from each of them.
- If your robots.txt sets a
Crawl-delay, Docket waits that long instead, up to 30 seconds between requests. - If your server answers 429 or 503, it slows down, and honours a
Retry-Afterheader. If refusals keep coming, it stops and says so in the report. - An audit reads 200 pages by default, and stops crawling new pages after 15 minutes whatever it has reached.
How robots.txt applies to it
An audit exists to show what search engines can reach, so Docket crawls what your robots.txt allows Googlebot to crawl. It also obeys any group written for it by name. To ask it to stay out of your site:
User-agent: Docket-SEO-Audit
Disallow: /Two limits, stated plainly rather than discovered. The address the person typed is still
read, because they asked for that page by name: once by the crawl, again in a browser engine
when the audit renders pages (which loads what that page itself loads, as any browser would),
and once as each crawler above unless the audit runs offline. Nothing else on your site is
requested: no other page, no sitemap, no llms.txt and no favicon. And the person running
the audit can switch robots.txt off, which exists for auditing their own staging sites. If that
matters to you, refuse the user agent at your server as well; that is the only rule a crawler
cannot choose to ignore.
The research survey
Now and then we read the robots.txt files of a published list of popular sites and report what they say, as counts on the Index. That survey does not use the audit name above. It sends this user agent:
Docket-SEO-Audit-Survey/1.0 (+https://docketseo.app/bot/; research survey: one robots.txt request per site, no crawling; contact support@docketseo.app)It asks each site for /robots.txt once. It asks for /llms.txt only
when the robots.txt it just read allows it, and if a real file answers there it makes one more
request, for a path that cannot exist, to tell a real file from a page that answers everything.
It works through the list about 16 different sites at a time, with one
request in flight per site and a short pause between the requests it makes to the same one. The
latest survey is described in
AI crawler blocking, August to October 2026,
which names no site. To be left out of the next one, add a group for
Docket-SEO-Audit-Survey with
Disallow: /, or write to us.
If it misbehaved on your site
Tell us, with the time and the address it requested, and we will find out why.
Common questions
Is Docket a search engine?
No. Nothing it reads is indexed or published. Each request belongs to an audit somebody ran from their own computer, and the results stay on it.
How do I stop Docket crawling my site?
Add a group for Docket-SEO-Audit with Disallow: / to your robots.txt. The person running an audit can switch robots.txt off for their own sites, so refusing the user agent at your server is the rule that always holds.
Why does Docket follow Googlebot's rules rather than only its own?
Because an audit reports what search engines can reach. A group written for Docket by name is followed as well.