LGTM at 9x: Your Merge Gate Was Built for Human Volume
Humans approve diffs they never read, and AI reviewers echo the author's stance, so merges now need test oracles the authoring model never touched.
An engineer at a mid-sized startup gave Gergely Orosz the current state of code review: “Everyone is playing the theater of doing reviews… give up on reviews and just stamp everything with LGTM.” Linear's telemetry in the same Pragmatic Engineer report shows the share of PRs reviewed only by AI agents rising. Change boards have decayed into signature collection before. A SOC 2 or change-management story claiming a second human read every change may already be fiction.
Every replacement check is tied to the author
The obvious fix is handing review to a model. It fails in ways that share one cause: the reviewer is correlated with the author.
- Shared blind spots. When one model family writes code and tests, their errors line up, and a test can faithfully encode the bug. Checking still costs real work. Jarred Sumner reports that at Anthropic, where AI writes nearly all the code and nearly all the tests, test effort still roughly equals code effort.
- Stance leakage. ByteByteGo's review of sycophancy research finds a model praises a design more once it learns the user wrote it, with the code unchanged. A PR description saying “I spent a week on this and think it's ready” hands the reviewer its verdict.
- Retry pressure. Agent loops retrying with “that's wrong, try again” push on preference alone, with no new evidence. The same research shows correct answers erode under repeated challenge until the model concedes.
OpenAI's April 2025 GPT-4o rollback put a price on this. The lab had no deployment evals tracking sycophancy, and favorable user feedback hid the regression. AINews notes Codex Auto-review is now free and doesn't count against plan usage, which makes a second AI reviewer cheap, though one from the model family that wrote the diff shares its blind spots.
Upstream review capacity is shrinking too
JavaScript Weekly reports that Sindre Sorhus has turned off outside pull requests because of AI contribution spam. SWC now accepts PRs only from trusted contributors. Both sit deep in most JavaScript dependency trees. A bug there means an issue and a wait, or a local patch through pnpm patch or patch-package. Test that workflow before it is needed.
The move: tier the gate by blast radius
| Path | Merge gate | Oracle the author didn't write |
|---|---|---|
| Low risk (docs, internal tools, isolated features) | Green CI plus agentic review, then auto-merge | Existing suite, plus a canary on deploy |
| High blast radius (auth, payments, data migrations, IaC) | A CODEOWNERS human who actually reads the diff | Mutation, property-based or differential tests against the old implementation |
| Any LLM reviewer | A prompt stripped of authorship and verdict language | A paired audit: the same diff with the stance flipped |
Hand the reviewer intent and constraints, such as “goal: cut checkout p99; must stay on Postgres”. Strip verdicts like “I wrote this” or “ready to ship.” Agent retries should carry evidence or nothing, meaning a failing test, a stack trace, or a schema error. Then harden deploys with SLO burn-rate gates and automated rollback. Unread code will reach production, so production has to catch problems quickly and cheaply.
Caveat: these field reports come from AI labs and AI-forward companies, which Orosz himself says run months ahead of the industry. Most curves will be flatter. Measure the local AI-only review share rather than assuming GitHub-wide growth.
Independence means the oracle was produced without the author's stance in context, whether the reviewer is human or model. The gate needs an oracle the author never touched.
What to do
Instrument code review this sprint: approval latency against diff size, the share of 300+ line diffs approved in under 2 minutes, and the share of PRs reviewed only by AI.
Prototype a risk-tiered merge policy this sprint. Low-risk paths auto-merge on green CI. Auth, payments, data migrations and IaC require a CODEOWNERS reviewer plus mutation, property or differential tests.
Before your next hosted-model version bump, strip authorship and verdict language from LLM reviewer and judge prompts, then audit them with paired stance-flipped prompts.