Case study
Case study: an AI agent broke checkout during a one-line copy change
Illustrative scenario. The team and numbers below are a composite example, not a real customer. The problem and the fix are real.
TL;DR
- The task was one line. The agent changed four files, including a shared price formatter.
- Tests passed because nothing tested the formatter with zero-decimal currencies.
- A task scope, a lock on checkout totals and a pre-commit check would each have caught it.
The setup
A two-person SaaS team sells subscriptions in several currencies. A developer asks Claude Code: "Change the checkout button from 'Pay now' to 'Start subscription'." It's a five-second job, so they review the diff quickly, see the new label, and merge.
What actually changed
The diff touched four files. Alongside the label, the agent noticed the button component used a helper called formatPrice, decided it was "inconsistent", and simplified it to always show two decimal places. It also renamed a prop and tidied an import.
For customers paying in Japanese yen, a currency with no minor unit, the checkout total now showed and charged the wrong amount. Support tickets arrived the next morning.
Why nobody caught it
- The review focused on the task. The reviewer checked that the label changed, not what else changed.
- Tests passed. The formatter's tests only covered USD and EUR.
- The rules file said "keep changes minimal". That's guidance. Nothing checked it.
Three guardrails that would have stopped it
1. A task scope
stet scope set "Change checkout button label" --allow "src/checkout/PayButton.tsx"
stet guard
✗ 3 files changed outside the task scope:
src/lib/formatPrice.ts
src/checkout/CheckoutForm.tsx
src/checkout/index.ts
The out-of-scope formatter change is flagged before commit, not found by customers.
2. A lock on what matters
stet lock add "Checkout totals are correct in every currency" \
--cmd "npm test -- tests/checkout" --protect "src/lib/formatPrice.ts,tests/checkout/**"
Now any change to the formatter is flagged, and the checkout tests run after every agent change. Adding a yen case to those tests closes the gap that let this through.
3. A check the agent can't skip
stet hook install # blocks out-of-scope commits locally
stet ci # runs guard and check on every pull request
The lesson
The agent wasn't careless. It was helpful in a way nobody asked for. Small tasks are where this hurts most, because nobody expects a one-line change to need a careful review. Make the scope explicit, lock the money paths, and let a check do the boring part of the review.
FAQ
Why did a one-line change break production?
The AI agent changed more than the task needed: it "simplified" a shared helper used by checkout. The review focused on the requested change, and the tests did not cover the case that broke.
How do I stop an AI agent from changing shared code during a small task?
Set a scope listing the files the task may change and run a check such as stet guard before commit; any other file the agent touched is flagged.
Which parts of an app should be locked against AI changes?
Anything that moves money, handles sign-in or permissions, or that customers depend on daily: checkout and billing, auth, exports and public APIs.
Put guardrails on your AI coding agent
Stet is a free, open-source CLI and MCP server for Claude Code, Cursor, Codex and Copilot. It runs locally: no account, your code never leaves your machine.
npx stet-cli init · Install · Setup guide · GitHub