How to gate a deploy on SEO regressions

A deploy that quietly removes your title tags does not break the build, does not throw an error, and does not show up in any dashboard until rankings move weeks later. docket diff audits two URLs — usually production and staging — and exits non-zero when the second one introduced something the first did not have.

docket diff https://example.com https://staging.example.com --fail-on high

What it actually reports

We ran it against a staged regression: a copy of a four-page site with every <title> stripped, which is what a template edit does when somebody moves the head block. The output, verbatim:

1 regression(s) — introduced by the candidate:
  HIGH     onpage.title_missing             new       4 pages have no title tag

3 improvement(s):
  MEDIUM   onpage.title_short               fixed     4 titles are very short
  MEDIUM   local.no_geo_in_titles           fixed     No page title mentions a location
  LOW      cvr.message_mismatch             fixed     3 pages promise one thing in search…

Read that again, because it is the whole argument for gating on a diff rather than a score. Deleting every title tag registered as three improvements. The checks for short titles, for titles missing a location, and for titles that do not match the page all stopped firing — correctly, since there is no title to be short or wrong.

And the overall score did not move at all — identical before and after, to the decimal. A gate watching the number would have let that deploy through, and a report showing three fixes against one issue would have looked like a decent week.

The exit codes

Measured, not quoted from a manual:

• 2 — a regression at --fail-on or worse. Fails the build.
• 0 — nothing introduced.
• 0 — with --fail-on never, which reports without failing. Useful for a first week of watching before you let it block anything.
• 1 — the audits are not comparable, and Docket refuses to judge.

That last one is the important one

Point it at a four-page baseline and a one-page candidate and it says:

These audits crawled very different numbers of pages (4 then 1), so per-issue comparison would be misleading. Re-run with the same page limit for a like-for-like changelog.

It exits 1, not 0. Refusing to judge is not a pass.

This matters more in CI than anywhere else. If staging is behind basic auth for half the crawl, or a sitemap has not generated yet, the candidate crawl is smaller — and a naive comparison reports every page the crawler never reached as a brand-new issue. You would get a wall of red on a deploy that changed nothing, chase it for an hour, and then start ignoring the gate. The failure that teaches people to ignore a gate is worse than the gate not existing.

The other half of the same decision: both sides are crawled with identical settings. There is no per-side page limit, because two crawls configured differently are not a comparison.

Why the default bar is lower than for a plain audit

docket audit defaults to failing on critical. docket diff defaults to failing on high, and the reason is that a regression is somebody's deploy. An issue your site has had for two years is a backlog item; the same issue arriving this afternoon is a change that just happened and can be reverted while the person who made it still remembers what they did.

In a pipeline

- name: SEO regression gate
  run: |
    docket diff "$PROD_URL" "$STAGING_URL" --fail-on high -n 200

Docket's Linux CLI is a single binary with no runtime to install. The rest of the CI story — SARIF for code scanning, JUnit for test panels, the GitHub Action — is here.

Download Docket

Common questions

How do I stop a deploy that breaks SEO?

Run docket diff with your production URL and your staging URL in the pipeline. It audits both with identical settings and exits 2 when the candidate introduced a regression at the severity you set, so the build fails before the change reaches production.

Why not just fail the build on the audit score?

Because a score can hide a regression. Stripping every title tag from a test site left the score unchanged and registered three improvements, since the checks for short, mismatched and location-free titles all stopped firing — and the overall score did not move at all. The diff still reported the high-severity regression.

What happens if staging is partly unreachable?

Docket refuses to compare crawls that reached very different numbers of pages, and exits 1 rather than 0. Refusing to judge is not a pass. Without that, every page the crawler could not reach would be reported as a brand-new issue and the gate would be ignored within a week.

Why does docket diff fail on high when docket audit fails on critical?

Because a regression is a change somebody just made. An issue a site has had for two years is a backlog item; the same issue arriving this afternoon can be reverted while the person who made it still remembers what they did.