The Bug Is in the Report
Checking what agents claim about their own code
In one run of seven coding-agent tasks, review confirmed 26 serious bugs. Eight of them contradicted something the agent had written about its own code. The claims appeared in reports, code comments, pull request descriptions, and the header of a script whose job was checking the work, and each was specific: an error could only be swallowed in one place, production code could not reach a certain database path, a checker would reject any new bypass. In several cases the agent had already looked at the evidence that contradicted its claim. In others it drew a broad conclusion from a narrow search.
Every check the agents shipped came with a negative control showing it could fail, because their task contract required one. The sentences they wrote about the code had no equivalent step. A general rule in the standing instructions asked for the same discipline without naming a step. The retrospective of the run estimated that checking any one of the eight sentences would have taken under a minute.
One of the Eight
One task added a guard to a family scheduling app. If a database write crossed from one household’s data into another’s, the app would refuse it by throwing an exception. The agent’s report said there was one place in the app where a failed write step could be swallowed, and it told me to search the logs for that code path’s error message after deploying. An empty search would mean everything was working.
The new exception inherited from a base type the app already caught in eleven places. Eight of those catch blocks wrapped database work, and all eight turned the error into an ordinary not-found or conflict response. A refused write would reach the user as a 404 or 409 and leave nothing in the log, so the search the report recommended would have come back empty however many writes were refused. The agent had searched the code for database saves but never for catch blocks. Its “one place” came from the task’s specification, which mentioned a single swallow site, and the agent generalized from it. The first version had already been deployed by the time review found the problem.
In two other cases, the contradicting evidence was already in front of the agent. One wrote that production code could not reach an unfiltered database path, repeating a sentence from the design notes almost word for word. Its guard scanned source code one line at a time, as the agent’s own comment on the guard said, and a constructor call split across two lines would get past it. That agent added more than 1,600 lines of C# without a compile error, and its claim about what the scanner prevented still failed review. Another agent switched its checker from matching exact lines to counting bypasses per file. Within three minutes of deciding on that switch, it wrote that any new, unrecorded bypass would fail the build, although replacing one bypass with another in the same file leaves the count unchanged.
The Rule I Already Had
My agents’ standing instructions already said that a claim should name the observation that would disprove it, and that a claim that can’t be checked should be labeled an assumption. An audit of the run found that instruction loaded in all fourteen sessions. Across eleven implementation sessions it produced one assumption label, and none of the eight contradicted claims showed any visible sign of it.
The line-scanner agent’s task specified several checks. For one of them, the task description told the agent to state the check’s limits, and the agent did. For the line scanner, the description had no such line, and the agent stated no limit and wrote that the path was unreachable. The audit found the same pattern across the run: the instructions agents visibly followed were the ones that named a concrete action at a known point in the work.
What I Changed
I turned the rule into a step and added a report section that a script checks for.
The step is now part of the same contract rule that requires negative controls. It reads, in part:
Every cannot / always / only / never / the one place / refuses anything else you write in a comment, PR body, pragma or report carries the one-line command that would refute it, run and quoted, or the word ASSUMPTION.
The report section is a required Debrief, repeated in the pull request description, with two headings:
- Forks I decided that the brief did not settle. Each choice the agent made on its own, the alternative it didn’t take, and what would show it chose wrong.
- Claims no control measures. Every sentence in the diff, comments, pull request or report that asserts a boundary (“only”, “every caller”, “cannot”, “is clear”) and has neither a quoted refuting command nor an assumption label.
“None” is an acceptable answer under either heading. A missing heading fails a check that runs before the report is accepted, so the section can’t be skipped quietly.
What the Next Two Runs Showed
The next two runs used the new instructions, with six tasks and three tasks. The rows below don’t count the same things, so read the table as a description of the practice rather than a before-and-after comparison:
| Run | Findings counted | Flagged by the agent before review | Contradicting the agent’s own sentence | Assumption labels |
|---|---|---|---|---|
| First | 26 major, full review | None in the agents’ working notes; 6 more named in reports, mostly carried over from the task instructions | 8 | 1 in 11 sessions |
| Second | 12 major or should-fix, mostly a lighter two-model review | 3 in the Debrief | 3 | 31 in 6 reports |
| Third | 10 should-fix, lighter review | 2 in the Debrief | 2 | 14 in 3 reports |
The later label counts are word counts, so a claim labeled twice counts twice, and the samples are small. What the later rows do show is that the Debriefs listed choices and claims that review then confirmed as defects, about a quarter of the findings in each run. Claims that contradicted the code still appeared in both runs.
The Debrief did not stop an agent from submitting a check it had already shown could not fail. In the second run, one agent added a check that treated a result as complete only when its stop reason said the output finished normally. Both code paths that produced results set that same stop reason, including the path for an interrupted stream, so the check could never fail. The agent wrote down that both paths set it, in its own assertions section, listed the design choice in its Debrief, and submitted the check for review anyway. Review caught it before merge.
One task in the third run added a clause to the rule for this case: if your own evidence shows a check can’t detect the defect it targets, fix the check or ask for a decision. The agent that wrote that clause also submitted a checker that didn’t enforce a limit the instructions stated, which review caught. I haven’t yet measured whether the clause helps.
If You Want to Try This
In my runs, three things seemed to matter: the specific trigger words, a place in the report where unverified claims have to be listed, and a mechanical check that the place exists. A principle in the standing instructions was not enough. Claims in the task specification need the same treatment, since two of the eight came from there.
When reviewing, read the Debrief first. It lists the choices and claims the agent itself wasn’t sure about, and in the two later runs about a quarter of the confirmed findings were already listed there.
Before changing anything, take the last report an agent wrote for you, find three sentences that use only, always, or cannot, and write the smallest check that could prove each one wrong.