AI code review tools shifted the bottleneck, they did not remove it

AI code review tools were supposed to collapse the time between opening a pull request and merging it. For a narrow set of checks, they have. But for the core job of a reviewer — deciding whether a change is actually correct and safe — most teams running these tools at scale report trading one bottleneck for another: the time once spent reading a diff is now spent deciding which of the bot's comments are worth reading.

Adoption has outpaced evidence of effectiveness

Stack Overflow's 2025 developer survey found 47% of professional developers had used AI-assisted code review in the prior year. JetBrains' survey from the same cycle put the figure at 44%. Those numbers climbed fast — both surveys showed roughly a doubling from two years earlier. Adoption, in other words, is no longer the interesting question. Whether the tools are actually catching the bugs that matter, without burying them in noise, is.

The false-positive problem is real, and vendor numbers should be read skeptically

A March 2026 benchmark from Signal65, which tested five commercial review tools against real historical bugs across six open-source repositories, credited CodeRabbit with 95.88% precision. That number comes from a vendor-commissioned study and should be weighted accordingly — precision figures from benchmarks designed around a single sponsor's strengths rarely survive contact with an unrelated codebase.

Developer-written accounts tell a messier story. Independent write-ups circulating among engineering teams describe unconfigured AI review tools producing false-positive rates in the 15 to 25 percent range on typical pull requests, dropping to roughly 7 to 12 percent only after teams spend weeks tuning the tool to their specific codebase, style guide, and known-safe patterns. That tuning cost is the part vendor pitches tend to leave out.

Where the tools genuinely work

The clearest wins are narrow and mechanical: flagging hardcoded secrets, catching SQL injection patterns, enforcing a style guide, and spotting the specific category of null-pointer and off-by-one errors that show up constantly in diffs. These are pattern-matching problems, and LLM-based tools are good at pattern matching across a diff's full context in a way that a linter is not.

The clearest failures are in judgment calls — whether a change actually fixes the underlying bug versus papering over a symptom, whether a new dependency is a reasonable tradeoff, whether a refactor preserves behavior that isn't covered by tests. These require understanding intent, and intent is exactly what a diff, stripped of the conversation that produced it, does not reliably communicate to a model any more than it does to a human skimming too fast.

What actually reduces the bottleneck

Teams that report a net time savings, rather than a wash, share a few practices. First, they scope the AI reviewer to a fixed category of checks — security patterns, style, and known bug classes — rather than asking it to review correctness broadly. Second, they treat the first month of deployment as a tuning period, feeding back false positives until the noise rate drops, rather than expecting out-of-the-box precision. Third, and most consistently cited, they keep a human as the final approver on anything touching authentication, payments, or data deletion, regardless of what the AI reviewer says, because those are the categories where a false negative is expensive enough that the triage cost of extra scrutiny is worth paying.

The pattern across all three practices is the same: AI code review works as a filter that narrows what a human needs to look at carefully, not as a replacement for that final look. Teams that deployed it expecting the latter are the ones now reporting that review takes just as long as it used to — the time has only moved from reading the code to reading the bot.