Short answer
A model reviewing its own code shares its blind spots, prefers its own output and rarely self-corrects without outside feedback. A useful reviewer differs in model, context and information: it reads the raw diff and evidence it did not produce, never the writer's reasoning.
Almost every multi-agent coding setup I see has a step called “review”. Often it is the same model, in the same session, asked to look over what it just wrote. It finds two typos, praises its own architecture and reports that the work is ready. Then a person opens the app and the main button does nothing.
The idea behind a review step is sound. The mistake is assuming that any second pass counts as a review. A second pass by the same mind, with the same context and the same incentive, is closer to proofreading your own essay right after writing it. You read what you meant, not what you wrote.
Why self-review fails for coding agents
There are three separate reasons, and each one has research behind it.
The first is that models are bad at correcting themselves without new information. Huang and colleagues at Google DeepMind tested self-correction on reasoning tasks and concluded that LLMs struggle to self-correct their responses without external feedback, and that performance sometimes got worse after the model “corrected” itself. A model asked “are you sure?” often changes a right answer to a wrong one, or defends a wrong one with more words.
The second is self-preference. Panickssery, Bowman and Feng showed that models such as GPT-4 and Llama 2 can recognise their own outputs above chance, and that the more a model recognises its own text, the more it scores that text higher than others’. A reviewer that likes its own handwriting is a weak reviewer.
The third is the most important one for anyone designing a pipeline: correlated errors. If two reviewers make the same mistakes, adding the second one buys you nothing.
Correlated errors, and why a different model isn’t enough
Kim and colleagues evaluated more than 350 language models and found substantial correlation in their errors. On one leaderboard dataset, when two models both got a question wrong, they picked the same wrong answer 60% of the time. Shared architecture and shared providers made it worse. The uncomfortable finding was that larger, more accurate models had highly correlated errors even across different architectures and providers.
That changes how I think about independence. Switching from one frontier model to another helps, because each lab trains and tunes differently, and in practice they do catch each other’s mistakes. But they read similar internet text and learned similar habits, so model diversity is only one layer. The other layers matter as much:
- Different context. The reviewer starts fresh. It has not spent an hour building up the writer’s assumptions.
- Different information. The reviewer sees the diff, the acceptance criteria and the output of commands that the harness ran. It does not see the writer’s plan, its reasoning or its summary.
- Different incentive. The reviewer’s job is to find what is wrong. Its success is a finding, not a finished task.
Here is how those layers combine. The labels are positions in a design space, not measurements.
How independent is your reviewer?
Data as a table
| Item | Information it shares with the writer | How different the model is |
|---|---|---|
| Same chat review | Everything | Same model |
| Same model new session | Only diff and evidence | Same model |
| Other model reads plan | Everything | Other provider |
| Other model, raw diff | Only diff and evidence | Other provider |
The top-left corner is a common trap. Teams wire up a second provider, then feed it the writer’s summary of what it did. The reviewer agrees with a story instead of checking a diff.
Judge biases that show up in code review
Research on LLM judges, mostly done on chat answers, lists biases that reappear when a model reviews code. Zheng and colleagues, in the paper that named “LLM-as-a-judge”, measured three of them.
Position bias: judges prefer whichever answer they see first. In their test, even GPT-4 gave the same verdict after the two answers were swapped only 65% of the time. In code review, this shows up as anchoring on the first file in the diff.
Verbosity bias: longer answers win. In code, this looks like a reviewer impressed by a large refactor with extensive comments, while a two-line fix that solves the actual problem gets less attention.
Self-enhancement: GPT-4 gave its own answers a win rate about 10% higher than humans did, and Claude-v1 about 25% higher, although the authors were careful to say their data could not prove the effect. Combined with the self-recognition result above, it is enough for me to never let a model grade its own work when the grade decides anything.
I cover how to run model judges without being fooled in LLM-as-a-judge for code.
What a real reviewer loop looks like
A writer and a reviewer, a structured way to pass findings back, and a limit on how long they can argue.
A writer and an independent reviewer
- Person
- Agent
- Automated check
Steps as text
| Step | What happens |
|---|---|
| 1. Criteria | Person — written before code |
| 2. Writer model | Agent — plans and edits |
| 3. Harness runs checks | Automated check — build, tests, exit codes |
| 4. Reviewer model | Agent — diff, criteria, evidence only |
| 5. Accept or park | Automated check — round limit reached goes to a person |
| Loop | findings go back: back from step 4 to step 2 |
The details that make it work:
- The reviewer receives the diff, not a description of it. If the change is large, that is a reason to split the change, not to summarise it. See small diffs for AI agents.
- The harness, not the writer, runs the build and tests and records exit codes. The reviewer reads those results. The reasoning is in evidence over claims.
- Findings come back in a fixed shape: file, line, what is wrong, how to verify the fix. Free-form criticism produces free-form arguments.
- There is a ceiling on rounds and a rule for “no progress”. Two models can disagree forever. When they do, a person decides, which is a good use of a person.
- The writer may reject a finding, with a reason. A reviewer that can’t be argued with drifts into style policing.
Disagreement before code, not after
Where you put the reviewer matters as much as who it is. I measured this once on a public task: rebuild a mobile game from a video of someone playing it.
The end-of-run review roughly halved the bugs but made the run longer, 3.7 to 4.2 hours, because the writer had already committed to a plan and the review rounds went into pulling it back. When two models first wrote independent approaches without seeing each other’s, and only material disagreements were raised, the run took 1.5 hours and the game had no broken mechanics. It is one task with counts made by me, so treat it as a direction, not a law. The full setup and caveats are in the benchmark article.
The reason it works fits the research above. Independent positions written before any code are the closest thing to uncorrelated opinions you can get from two models. Once the code exists, the reviewer is anchored by it.
Bad and good setups compared
| Setup | Why it fails or works |
|---|---|
| “Review your own code” in the same chat | Same context, same blind spots, self-preference |
| Same model, fresh session, sees the writer’s summary | Fresh context, but it reviews the story, not the change |
| Different model, sees plan and summary | Different habits, still anchored by the writer’s framing |
| Different model, sees diff, criteria and harness evidence | Independent in model, context and information |
| Two models write approaches before code, then compare | Disagreement surfaces while it is still cheap |
I run the reviewer as a different model from a different provider in my own pipelines, and I would not go back. It is not that one model is smarter. It is that two models with different training disagree in useful places, and a pipeline only needs one of them to notice.
Questions
Can an LLM review its own code?
It can catch surface mistakes, but it is a weak reviewer of its own logic. Research shows models rarely self-correct reasoning without outside feedback and tend to prefer their own outputs, so use a separate model with fresh context for any review that decides a merge.
Is using two different AI models better for code review?
Usually yes, because different providers train differently and miss different things. Errors are still correlated across large models, so also give the reviewer different information: the diff and real test results, not the writer’s summary.
What is a multi-agent coding workflow?
It is a setup where several agents with different roles work on one task, typically a planner, a writer and a reviewer, with a harness that passes work and findings between them. Its value comes from the roles being independent, not from the number of agents.
Should the reviewer agent see the writer’s reasoning?
No. The reviewer should see the acceptance criteria, the diff and the output of commands it did not run. Showing it the writer’s plan or summary anchors it to the writer’s view and removes most of the benefit of a second reviewer.
