Christine Seeman’s team hit the point where a pull request that gitignored a single directory — one line of code, ten characters — came with a description running 1,430 characters, which engineers, she wrote in July , were still expected to read in full. More broadly, she described a bot commenting on a pull request and a second bot replying to it — comment threads a human still had to read in full. That’s the noise AI code review can generate, and it’s what Wealthfront’s engineering team set out to fix by rebuilding its internal reviewer this year. The result, published August 3, spends about $4 and ten minutes of compute on nearly every code change, and counts it a success when the AI has nothing to say — the opposite of how every AI review product on sale is still pitched, on how much it catches, not how little it says.
Three vendors argue so most pull requests hear nothing back
The tool, Iris Code Review, runs Anthropic’s Opus 4 as lead reviewer. When Opus flags something that might be a real problem, it doesn’t just post the comment — it spawns two more agents, one on GPT-5 arguing the concern is real, one on Gemini 3.0 Flash arguing it isn’t, and Opus weighs both before a human sees anything, per Wealthfront’s writeup . Three vendors’ worth of model time gets burned largely to talk itself out of speaking. The team wanted “to pay $20 per review and were happy to wait 15 minutes or more” for fewer wrong comments; what it built runs cheaper and faster while cutting the noise anyway. Wealthfront bought precision with compute — spending three models’ worth of time to produce fewer comments, not more — and treats that trade as the win. The review stays advisory, not a gate: “the peer review is blocking but the AI review is not,” a division of labor that fits how this magazine found human review relocating rather than disappearing even as AI takes the first pass.
Noisy AI review already has a measured fate: getting ignored
That bet has evidence behind it. A mining study of 31,073 review-and-feedback pairs across 239 GitHub repositories, Is Agentic Code Review Helpful? , found agentic review comments accepted only 36.4% of the time and rejected 56.3%, mostly for being false positives, redundant, or out of scope. A separate MSR 2026 study of pull requests scraped from AIDev measured the same flood Seeman had already described from inside one team’s inbox : across 3,109 pull requests that reached the commented-review state, 60.2% of closed AI-only-reviewed ones carried a “signal” rating between 0 and 30%, and such pull requests merged at 45.2% against 68.37% for human-only review. Two research teams, watching different populations, measured the same fate: low-signal review doesn’t get corrected — it gets tuned out.
Wealthfront bought precision with compute — spending three models' worth of time to produce fewer comments, not more.
That’s the cost Seeman named with her team’s own rule of thumb, borrowed from her CEO: “return on attention,” the idea that every word asked of a reader has to be worth what it costs them. If you’re the engineer opening a pull request and finding eight bot comments waiting, this is what’s being decided: whether your reviewer is tuned to fill that thread or stay out of it, and whether you’re allowed to mark a comment “actively harmful” and have that count against the tool.
Confident reviewers approve bugs just as fast as good code
CodeRabbit, whose own comments are part of what the mining study measured, made the case for the opposite premise months earlier, in a June post titled “We watched developers approve bugs in 30 seconds” : at 30-second review-speed competitions run at the app.js, JS Nation and React Summit conferences, “a chunk of the room would confidently wave it through,” and “the ‘obviousness’ of a bug turned out to have almost nothing to do with whether people caught it at speed.” It’s a vendor arguing for its own product, so treat it as framing, not evidence — but the logic holds: a quieter reviewer that routes more code past a human who misses obvious bugs at a glance just relocates the failure, the same blind-spot problem this magazine traced through GhostCommit’s prompt-injection attacks , where reviewers and code-writing agents don’t see the same things.
Wealthfront’s own numbers don’t fully settle it either. The post reports a large drop in comments rated 1 or 2 — “actively harmful feedback” sits at the bottom of its five-point scale — and only a “small increase” in the 4s and 5s that mark a caught bug: the right direction, but modest where it matters most. What gets measured throughout is engineer satisfaction with a comment, self-reported inside Wealthfront, while the mining studies measure acceptance or rejection in the open — neither counts a defect that reached production anyway. Read across from what’s actually published, quieter review looks like better-tuned review, not necessarily review that catches more; the number that would settle it, an escaped-bug baseline, isn’t in the post.
What the vendor pitch and Wealthfront’s writeup skip past, in different directions, is the same fact: a rating scale starting at “actively harmful” admits something no vendor benchmark does, that an AI reviewer’s output can score below zero. Every review tool on sale is measured by what it finds. Iris is measured, one comment at a time, on whether it should have spoken at all — and the engineer holding the pen on that judgment, not the model doing the arguing, is the bottleneck nobody in this category has priced in yet.
