Merge Rate Became AI Coding's Scoreboard, and It Doesn't Agree

Ask an engineering leader in 2026 why the team standardized on one coding agent over another, and the answer increasingly reduces to a single number: how often its pull requests get merged. Merging is the moment code stops being a proposal and joins the codebase, so the rate looks like it measures competence directly. Vendor dashboards report it, and a fast-growing shelf of academic papers built on the public AIDev dataset — Yuan Li, Zhuoyi Zhang and Ahmed Hassan’s catalogue of 932,791 agentic pull requests across 116,211 repositories — treat it as the obvious yardstick for comparing agents. The trouble: four groups measuring roughly the same thing don’t agree, and the size of that disagreement is the real story.

A study published July 23, 2026 by Iren Mazloomzadeh, Mohammad Mehdi Morovati and Foutse Khomh, working from a curated slice of that dataset — 9,428 agentic pull requests from five agents across 489 Python repositories with 100+ stars — found Claude Code merging 84.3% of the time and Devin just 43.0% , a gap wide enough to look decisive. Months earlier, Giovanni Pinna, Jingzhi Gong, David Williams and Federica Sarro ran the same five agents through essentially the same dataset and got a different order: Codex first at 77.9%, Cursor second, Claude Code third at 71.9%, Devin last at 61.6%. Zoom out further and LinearB’s 8.1-million-PR benchmark across 4,800 teams puts AI-assisted PRs merging within 30 days at 32.7%, against 84.5% for human ones — a figure so much lower it barely resembles the same metric.

The dataset is one thing, the sample is four different things

The obvious read is that one paper is wrong. The more useful read is that each is measuring a different corpus and calling it the same word. Mazloomzadeh’s team sampled documentation, dependency-management and testing tasks as the most common agentic work — the easy end of the distribution, where acceptance runs high almost by construction. Pinna’s group stratified explicitly by task and found documentation PRs accepted at 82.1% against 66.1% for new features, concluding task type predicts acceptance better than agent choice in most categories. LinearB isn’t even counting the same population: it tracks AI-assisted PRs opened by human developers, inside a fixed 30-day window, where a PR that merges on day 45 counts as a miss. None of that makes the numbers dishonest — it makes “merge rate” a label stretched over four experimental designs, and a leaderboard pasting those percentages together is comparing repo policy and task mix as much as it’s comparing agents.

A rejected fix rarely comes with a reason attached

The clearest sign that maintainer attention, not code quality, drives the number sits in a companion paper on the same AIDev substrate. Studying 3,225 agent-authored fix pull requests, Mahmoud Abujadallah, Ali Arabat and Mohammed Sayagh at École de Technologie Supérieure found 46.41% rejected — and when they traced why, 49.3% carried no documented reason at all. Another 17.3% died from plain inactivity, and outright technical failure — a wrong fix, a failing check — explained well under a fifth of rejections. The median rejected PR still carried real churn, 81 to 293 changed lines a maintainer had to at least skim before letting it lapse. A rejection rate built mostly out of silence and timing isn’t describing what the agent wrote; it’s describing whether a maintainer had a free afternoon that week — the same reading this magazine took from Amazon and Meta quietly abandoning their internal AI-usage leaderboards , where the score tracked what got gamed, not what got shipped.

That doesn’t make the divergence empty noise, and the fair version of this argument grants the counterpoint its due. Some of it really is definitional rather than damning: LinearB’s window-bound, human-assisted framing and the arXiv papers’ agent-authored, star-filtered framing were never going to converge. Pinna’s team also found something that looks like a genuine trend rather than noise — Devin’s acceptance climbing 0.77 points per week, a longitudinal signal a snapshot leaderboard can’t see. And the pessimistic reading, that low merge rate implies bad code, doesn’t hold up either: Mazloomzadeh’s team explicitly checked and reported that merged agentic PRs show “comparable or lower defect proneness than human PRs, with mostly non-significant differences.” Low acceptance and low quality are not, on this evidence, the same claim.

What actually distinguishes the work is what happens after merge

Which leaves the more useful question untouched by any merge-rate number: what happens to code once it’s in. Faros AI’s two-year telemetry sweep of 22,000 developers found PR merge rate climbing 16.2% under heavy AI adoption while median time in code review rose 441.5%, PRs merging with no review at all rose 31.3%, and code churn rose 861%. As the report puts it, “throughput measures what was shipped, not what survived,” a pattern of rework running parallel to GitClear’s own findings on rising duplication and shrinking legacy maintenance since 2023. Churn, defect escape and review depth measure the code; merge rate measures the room around it, and that room has lately been busy, understaffed and inconsistent about writing anything down. A team picking an agent off a merge-rate leaderboard isn’t reading the agent’s competence. It’s reading its own review queue back to itself, dressed up as a stopwatch.