You check the AI’s work the way everybody checks it: open the component in the preview pane, at whatever width your laptop happens to be sitting at. It looks right. That’s the entire inspection. A benchmark published 12 August 2026, “Does It Render Everywhere?” , rendered 203 AI-generated webpages across nine real browser-and-device combinations — Pixel and iPhone, Chrome and Safari, two desktop operating systems — and found 68% broke somewhere. That’s 1.7 times the 40% failure rate the same researchers measured on human-written pages from the same source datasets. The AI writes code that says it works on phones. Most of the time it doesn’t, and you cannot tell by reading it.
The wrapper accounts for the spread, not the model
The 68% average is the headline, and it’s the wrong number to remember. The paper’s authors — Guo, Xiao, Sun and Huo — tested eight generation methods; the per-tool spread runs from 26% for Vercel’s v0, better than the 40% human baseline, to 100% for a raw call to GPT-5.1. Cursor, the other commercial product tested, failed 79% of the time on the paper’s combined figure, comparable to v0’s 26% — worse than three of the six non-commercial baselines tested, better than the other three. Two products on comparable models land 53 points apart: what predicts survival on someone’s phone isn’t the model generating the page, it’s what happens to that output before a person sees it. As Vercel and Figma’s race to push AI prototypes into production already suggested, the competition among these tools has moved to the wrapper, not the model inside it.
| Category | Failure rate (%) |
|---|---|
| v0 | 26 |
| Cursor | 79 |
| GPT-5.1 (raw) | 100 |
| Human baseline | 40 |
The failures pass because they look right
That wrapper problem would be easier to catch if broken pages looked broken. Mostly they don’t. The paper reports that “although most MLLMs incorporate responsive design patterns into the generation, they fail to properly implement these codes” — the media queries and fluid units meant to adjust layout for screen size are present, and wrong anyway. The largest failure category, 42.1% of all defects, is what the authors call “Shrink-to-Fit”: pages that “shrink directly to fit the target screen with too small fonts.” Nothing overlaps, nothing spills off the edge, every element sits where it belongs — at a size nobody can actually read. It’s the failure most likely to survive a desktop screenshot review intact, because structurally, nothing is wrong.
The paper never watched a design engineer approve a pull request; what it measured is annotated screenshots and post-render DOM trees, not a review process. But read across to how a design-to-code handoff actually gets checked — a diff read, a glance at the preview pane — and the gap the study found is exactly the one those two checks are built to miss. A related benchmark found the same mismatch in AI design tools’ stated rationale for a screen , absent from the underlying code more than a quarter of the time.
A diff confirms the pattern is present, and a preview confirms it looks right at one width — neither catches a defect that only appears somewhere else.
For a design engineer approving a generated component after that single glance, the odds aren’t reassuring: something close to two in three that the layout breaks on somebody’s actual phone. The fix isn’t reading the CSS more carefully — the code already says the right thing. It’s opening the component at three or four more widths before clicking approve, the way you’d size something with a tape measure instead of trusting how it drapes on the one body in the room.
The honest complication is in the benchmark’s own numbers
That complication is real. Six of the eight generators tested are raw one-shot model calls or academic pipelines nobody ships; strip those out and the two commercial tools split hard, 26% for v0 against 79% for Cursor — the same tool-mix distortion Pipeline traced in AI coding’s merge-rate benchmarks . Cursor’s own docs for its agent browser tool tout testing responsive designs, feeding screenshots back so the agent “actually sees the browser state as images” — a check-and-fix loop the study’s neutral, one-shot prompts never invoked. And the 40% human baseline cuts the other way: the pre-AI web these datasets came from was already broken four times in ten — a regression, not a new disease.
None of that changes what the render-level failures are, only how much of the average to pin on any one tool. Fixing it with one better look doesn’t survive contact with how Nielsen Norman Group’s Raluca Budiu frames judging AI output generally:
“One good output demonstrates that a system can perform a task. It does not show how often or how reliably the system will do so.”
Her prescription — test many inputs, several times each, and report a rate rather than a verdict — was written about chatbots, but it names the same gap between one preview glance and the nine-environment grid this benchmark ran.
The preview pane isn’t going away, and it shouldn’t — it’s still the fastest way to catch failures a diff can’t see either. What the benchmark actually recommends is smaller than a new tool: stretch the window, or open the component on an actual phone, before the fix that took thirty seconds to generate costs someone else an afternoon.
