The Docket crawler
If Docket-SEO-Audit is in your logs, somebody audited your site with
Docket from their own computer. There is no Docket crawler farm and no scheduled sweep of the web:
every request under this name belongs to one audit that one person started.
How it identifies itself
Every page request carries this user agent:
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0 Safari/537.36 Docket-SEO-Audit/1.0 (+https://docketseo.app/bot/)Pages that are also opened in a browser engine — to see what JavaScript builds — carry the same name and the same link back to this page. Docket never claims to be Googlebot or any other search engine's crawler.
How hard it pushes
- At most 4 requests at a time, with at least 0.25 seconds between requests to your site from each of them.
- If your robots.txt sets a
Crawl-delay, Docket waits that long instead — up to 30 seconds between requests. - If your server answers 429 or 503, it slows down, and honours a
Retry-Afterheader. If refusals keep coming, it stops and says so in the report. - An audit reads 200 pages by default, and stops crawling new pages after 15 minutes whatever it has reached.
How robots.txt applies to it
An audit exists to show what search engines can reach, so Docket crawls what your robots.txt allows Googlebot to crawl. It also obeys any group written for it by name. To ask it to stay out of your site:
User-agent: Docket-SEO-Audit
Disallow: /Two limits, stated plainly rather than discovered. The address the person typed is still requested once — they asked for that page by name — but nothing beyond it. And the person running the audit can switch robots.txt off, which exists for auditing their own staging sites. If that matters to you, refuse the user agent at your server as well; that is the only rule a crawler cannot choose to ignore.
If it misbehaved on your site
Tell us, with the time and the address it requested, and we will find out why.
Common questions
Is Docket a search engine?
No. Nothing it reads is indexed or published. Each request belongs to an audit somebody ran from their own computer, and the results stay on it.
How do I stop Docket crawling my site?
Add a group for Docket-SEO-Audit with Disallow: / to your robots.txt. The person running an audit can switch robots.txt off for their own sites, so refusing the user agent at your server is the rule that always holds.
Why does Docket follow Googlebot's rules rather than only its own?
Because an audit reports what search engines can reach. A group written for Docket by name is followed as well.