I wrote about the writing side of the sprint yesterday — the four overnight PRs, the /goal command, the one-line checkpoint clause that let me sleep. That post was about producing code fast.
This one is about the thing that makes fast writing not-terrifying: the harness that proves the code I just wrote didn’t accidentally change what the paying users see.
The four PRs all touch the same file
Every PR in the sprint modified the public checkout. Different features, same code path. Individually each one is small. Together they collide in a dozen files. And the checkout is where money lands, so I don’t get to say “probably fine” and merge on vibes.
The rule I put on the sprint before any of it ran:
With the feature flag off, this branch must be byte-identical to main.
— the sprint constitution
Not “close enough.” Not “no observable regressions.” Byte-identical. The same DOM under #root, the same pixels on the payment page, the same rows in the ledger after a real booking, the same notifications firing. Every PR in the sprint ships behind CHECKOUT_V2_ENABLED, and with that flag false, the customer-facing surface has to look exactly like what shipped last week.
That rule has a name in the repo. It’s called flag-off parity, and there’s a spec that enforces it.
What “identical” actually means
Two things have to hold, both compared against a reference captured from origin/main on the same machine, minutes apart, against the same seeded database.
1. DOM under #root, normalized. Everything Playwright can serialize — text, structure, classes, order, inline styles — must match exactly. The only things stripped are the six attributes React auto-generates that shift with tree depth: id, for, aria-controls, aria-labelledby, aria-describedby, aria-owns. And autocomplete, because I actually did add real <label for> and autocomplete wiring on the branch and want that improvement to count as a diff worth reviewing.
2. Full-page screenshot within 50 pixels. 1280×900 viewport, light color scheme, timezone pinned to America/Chicago, caret hidden, animations disabled. The 50-pixel floor exists because a byte-identical DOM will still differ by ~20 pixels of caret anti-aliasing between two identical renders. Anything visible to a human is thousands of pixels. The threshold sits well below “something changed.”
Six pages get captured this way: the grid page for a tenant with every product enabled, the private flex-charter product page before and after a priced selection, the public fixed-slot page before and after a selection, and the payment page after clicking Continue.
Two copies of the site, side by side
The harness runs both versions on the same laptop, minutes apart, under a shared-stack lock so nothing else in the sprint touches Redis or the local Postgres while it’s running.
The reference run is a fresh clone of origin/main. API on port 3104, web on 5184. It writes DOM captures and Playwright screenshots into e2e/tests/parity/__reference__/.
The candidate run is the branch, with CHECKOUT_V2_ENABLED=false. API on 3103, web on 5183. It reads the reference, and asserts equality.
Both runs share the same seeded fixture — a tenant with a grid flow, one public fixed-slot product, one private flex charter, deleted in afterAll. Both runs get their Novu calls intercepted by a NODE_OPTIONS=--require preload so no push notification ever leaves the machine.
If the assertion fails, Playwright dumps the first DOM offset that differs with context from both sides into __actual__/, which is gitignored. Read the message, look at the two files, decide: is this a real change, or a codegen artifact I need to normalize?
The version that goes all the way to money
There’s a second spec that doesn’t stop at DOM. It performs a real booking end to end — a fixed-slot booking and a flex-charter booking — through the legacy checkout, with Stripe’s 4242 4242 4242 4242 test card on a test-mode Connect account. Then it records everything a booking leaves behind:
- The reservation row.
- The quote version and every line item.
- The Stripe payment intent id and status.
- Every reservation event fired.
- Every Formance ledger transaction posted, with the exact debits and credits.
- Every Novu trigger call, captured by that same
NODE_OPTIONSpreload.
All of it gets normalized (ids, timestamps, provider ids swapped for shape-preserving placeholders) and compared against the reference. If the flag-off branch takes even one different ledger path than main, the spec fails.
The one policy that matters
There’s a single sentence in the parity README that keeps the whole thing from being theater:
A diff here with the flag off is a customer-facing change on a live operator’s checkout and must be either gated behind
capabilities.checkout_v2or justified in the PR body — never accepted by re-recording the reference from the branch.— Brett Ridenour
That last clause is the whole game. The most tempting failure mode of a golden-file test is: “the test is red, let me update the golden file.” That’s how you launder a regression through your own CI.
The rule bans it. If the parity spec is red, one of two things has to happen. Either the change gets moved behind the CHECKOUT_V2_ENABLED flag so it only affects operators who’ve opted into v2 — meaning nobody yet — or the change is justified in the PR body as an intentional customer-facing improvement (real accessibility wiring, a security fix) and the reference is re-recorded from main, not from my branch, after main gets that same improvement.
The reference is always a photograph of main. Never a photograph of what I want main to be.
Why an agent sprint needs this specifically
The last post celebrated the fact that a coding agent can produce four PRs while I sleep. That’s real. It’s also dangerous without a rail like this one.
An agent isn’t going to notice that it accidentally re-ordered two <div>s and pushed a rebuy button 40 pixels to the left. The /goal gates I wrote about yesterday check that migrations enable RLS and that endpoints get documented — they don’t check “did the pixels move.” The parity spec is what catches the class of regression the agent’s own gates are blind to.
Together they form a pair. /goal proves the new thing works. Parity proves the old thing didn’t break. Neither alone is enough. Both together let me push four PRs from a hotel and open GitHub the next morning without opening my banking app first.
Takeaway
If you’re letting an agent write refactors on a code path where money moves, the question isn’t “did the agent write good code.” It’s “can I prove, with a shell command, that the paying users see exactly what they saw yesterday.”
The answer here is a Playwright spec that boots two copies of the app, walks a real booking through Stripe test mode, and diffs the resulting ledger. It runs in about six minutes. It is the cheapest insurance in the repo, and it is the only reason I trust an overnight run near the checkout code at all.
Fast writing is a feature. Byte-identical parity is what makes it safe to ship.