Six Guardrails on My Unattended Bug-Fix Loop

The pitch is scary if you say it out loud. “For the next two hours, drive Chrome around my SaaS, look for anything broken, file it as a GitHub issue, and if it’s small enough, fix it.”

That’s what /autopilot 2h does inside the Freebo repo. It’s a Claude Code skill that runs a controller loop: a stress-test phase, a triage phase, a fix-attempt phase, a heartbeat, repeat until the wall clock hits the cap. It has caught real bugs I wouldn’t have found because I don’t click every page every day. It has opened draft PRs I’ve merged, and draft PRs I’ve closed.

The reason I can leave it running while I’m at a concert isn’t because it’s smart. It’s because of six rules pinned at the top of the skill file that describe what it is not allowed to do. If any of them evaporated, this would be a disaster instead of a tool.

The six rules, exactly as they’re written

Every one of these is a line I hit while building this. They’re not aspirational — they’re scar tissue.

1. Draft PRs only. Never merge. Never mark ready-for-review.
2. P2/P3 only get auto-fix. P0/P1 get filed as issues with human-review-required and skipped.
3. Never push to main. Always feature branches: auto/fix-issue-<N>.
4. Halt on broken baseline. If npm run typecheck OR npm run build fail at iteration start, write HALT.md and stop.
5. Halt after 3 consecutive fix failures.
6. Hard time cap. Check the wall clock at the top of every phase. No exceptions.

Every one of those does load-bearing work. Let me walk through the two that matter most and then the one that’s easy to miss.

Rule 2 is what makes it safe

The autopilot’s triage phase spawns a sub-agent that classifies every bug it finds by severity. That severity label is not decoration. It’s a fork in the road.

gh issue list --label claude-fix --state open --json number,title,body --limit 20

Only issues carrying claude-fix get considered by the fix loop. And only P2/P3 issues receive that label. P0/P1 receive human-review-required instead, and the autopilot’s fix loop refuses to look at them.

That is the whole safety story of an autonomous bug fixer. The kinds of bugs that should be automated are cosmetic misalignments, a wrong tooltip, an inconsistency in a status label. The kinds of bugs that should never be automated are the ones where a wrong fix creates a worse production incident than the bug itself — anything touching payments, permissions, auth, cross-tenant data. A bot that can’t tell the difference between those two categories doesn’t get to run unattended.

Rule 1 makes every failure recoverable

The controller never merges. It opens draft PRs and stops. Even when the fix passes typecheck, lint, build, and the whole test suite, the PR is still a draft. Even when a sibling skill reviews the diff and returns SAFE, still a draft.

That means the worst possible autopilot run — every P2 misdiagnosed, every fix technically correct but strategically wrong — produces a queue of draft PRs I can bulk-close in ten seconds. There is no in-flight harm. The blast radius is a stale branch list.

Rule 5 is the one I forgot the first time

The first version of the autopilot didn’t have “halt after 3 consecutive fix failures.” It ran a two-hour window and cheerfully attempted twelve fixes, all of which failed in variations of the same way — same test flake, same misleading error, same wrong path down the codebase. Zero commits. Zero PRs. Twelve wasted iterations.

That’s not just a cost problem. It’s a signal problem. If the loop can’t fix three things in a row, the environment is broken in a way the agent can’t see. Maybe main got a new lint rule. Maybe the test runner is downloading a Playwright binary on every attempt. Maybe the classification labeled a database migration as P3 (it wasn’t).

Three failures and it writes HALT.md, drops a final report, and stops. The failure state is loud. Silence-with-nothing-shipped is worse than a stop signal.

What one run actually produces

The controller writes everything to a run directory keyed to a timestamp: docs/autopilot/<run-id>/. Inside:

• START.md — when it started, the git SHA it started from, the baseline typecheck+build result
• STATUS.md — one heartbeat block per iteration (bugs logged, issues filed, PRs opened, consecutive failures)
• iter-N/findings.md — per-iteration bug log from the stress-test phase
• END_TS — the wall-clock deadline, on disk, so it survives context compaction
• FINAL.md — the wrap-up table: issues filed, draft PRs opened, failure reasons, next steps for me

That END_TS file is the small trick that lets the loop survive if Claude’s context auto-compacts mid-run. The controller reads END_TS from disk at the top of every iteration instead of trusting a shell variable, because after compaction there is no shell state. The deadline is a file.

The heartbeat lesson

There’s a rule that isn’t in the six but should be. The first autopilot version echoed every FIX_COMMITTED and FIX_FAILED result to the chat as it happened. Every 30 to 60 seconds, a line. My phone lit up like a group text. I could not do anything else.

The fix in the current version is one sentence in the skill: “Do not echo individual FIX_COMMITTED / FIX_FAILED messages to the user during the loop. Collect them in memory across the iteration; emit one combined table at the end of each iteration via the STATUS.md heartbeat.”

One line of output per iteration. Ten to fifteen iterations across two hours. A digest, not a live feed. Unattended shouldn’t mean “constantly asking for attention.”

1. T+0m
   Pre-flight
   Verify clean tree on main, baseline typecheck + build must pass. Otherwise HALT.
2. T+0–25m
   Stress-test
   Sub-agent drives Chrome around staging. Logs bugs to findings.md.
3. T+25–35m
   Triage
   Sub-agent classifies each bug. P0/P1 → human-review-required. P2/P3 → claude-fix.
4. T+35m–end
   Fix loop
   For each claude-fix issue: branch, /fix-bug-team, if all checks pass → draft PR. 3 fails in a row → HALT.
5. T+cap
   Wrap-up
   Write FINAL.md, open a draft PR against main with the run report itself.

What actually happens when I come back

I open the run directory. I read FINAL.md, which is a table of issues filed and draft PRs opened, with links. I look at the safety:review-needed PRs first — those are the fixes the post-fix reviewer flagged as UNSAFE, which are still draft, still on a feature branch. Some I merge. Some I close with a comment. Some become the seed for a real fix I write myself in the morning.

I’ve had runs open five draft PRs and had runs open zero. Neither is a failure state. A zero-PR run where the stress-test caught two P0s I now need to look at is a great outcome. The autopilot’s job isn’t to close bugs. It’s to bring things to my desk in a state where I can act on them fast.

The one line that pulls the whole thing together

Every unattended-agent post you read is either a demo of a toy problem or a warning about AGI. The pattern that actually lets me leave one running is smaller than either:

Write down what it can’t do, in numbered rules, at the top of the file. Test them one at a time by trying to violate them.

Rule 2 is the safety net. Rule 1 is the recovery net. Rule 5 is the smoke alarm. The other three are the fence posts.

If you’re building an autonomous loop that touches your own production system in any way, the interesting question isn’t which model you’re using. It’s what the smallest number is where it’s willing to stop.