Andy Budd has seen this failure mode before, just not manufactured at this scale. Writing in Smashing Magazine on 29 July 2026, the design consultant warned that “plausible design is dangerous. It looks coherent in a product review. It uses the right components. The spacing is fine.” His conclusion was blunt: “A lot of bad product decisions already survive because they look plausible. AI will produce more of them.”
Five days earlier, a benchmark called “Design Theater” — built by researchers led by Kashif Imteyaz, with Michael Muller and Saiph Savage among the co-authors — measured exactly how that plausibility gets manufactured. Prompt-to-UI tools such as ChatGPT, Claude, Firebase Studio, Vercel v0 and Bolt don’t just hand back an interface; they hand back a fluent account of why they made each choice. Scoring 120 generated interfaces against those tools’ own stated rationales, the researchers found more than a quarter of the explanations described something the code didn’t actually do — rising to 34% for functional requirements.
The explanation stays honest about anything a screenshot would catch
The gap is shaped like a screenshot. Across the 120 interfaces, the paper’s Thinking Fidelity Score averaged 0.75, and held highest on styling, at 0.81, and structure, at 0.79 — the layers you can verify by glancing at a screen. On functional tasks the score drops to 0.66, and on the specific functional UX principles named in each prompt, four of the five tools scored 0.06 or lower. Two independent human raters did the scoring, with agreement solid enough to trust the pattern (κ = 0.70 for that fidelity measure, κ = 0.90 for a second adherence score). The tools stay roughly honest about anything a still image would catch, and stop being honest about anything that requires pressing a key.
| Category | Thinking Fidelity Score |
|---|---|
| Styling | 0.81 |
| Structure | 0.79 |
| Functional | 0.66 |
The tools stay honest about anything a screenshot would catch, and stop being honest about anything that requires pressing a key.
The rationale has quietly become the review surface
Imteyaz and his co-authors are careful about what they actually measured: whether a stated rationale shows up in the artifact a tool produced, scored by comparing extracted elements across static screens. They explicitly did not test whether anyone reads those rationales, trusts them, or ships on the strength of them — that’s outside an artifact study’s scope. Read across from what they measured to Budd’s account of plausible work sailing through a product review, though, the picture completes itself: the rationale isn’t decoration bolted onto the interface afterward, it’s functioning as the review surface designers actually consult before they click through — the same mismatch Pipeline found in AI-generated documents that describe intentions the codebase doesn’t keep .
That matters because the write-up is worst exactly where a reviewer needs it most. A design engineer opening a build from this generation of prompt-to-app tools tends to read the explanation before clicking into the build itself — the fastest way to sanity-check dozens of generated screens against a prompt. But keyboard navigation, focus states and error handling are precisely the functional principles the benchmark found missing most often, so the review has to stop reading and start tabbing through the interface by hand, testing exactly what the narration was least reliable at describing. That lines up with what Nielsen Norman Group’s Raluca Budiu argues about evaluating AI output more broadly: “a single output is an example, not an evaluation,” and one good result never shows how reliably a system repeats it.
The benchmark’s own limits keep the headline number honest
The paper’s limitations section undercuts its most dramatic figure, and it deserves to be taken seriously rather than filed as a footnote. The benchmark constrained every tool to plain HTML, CSS and JavaScript with no component libraries — a choice that plausibly depresses functional scores on its own, since a tool that can reach for an off-the-shelf library gets keyboard-navigable tabs and focus states almost for free, and has to hand-roll them otherwise. The authors also measured artifacts only, not how stakeholders actually interpret or trust the rationale, and they’re explicit that five tools at one version is a snapshot of fast-moving systems, not a verdict. At three and a half weeks old, Design Theater has no published rebuttal yet, and no independent replication has surfaced either.
None of that changes what a quarter of the rationales already say about themselves — it only bounds how far the number should travel. Budd’s product review, the one where everything looks coherent, the components are right, the spacing is fine, was never really checking against the code. It was checking against a story about the code, told fluently by the same system that wrote it. Until that story earns the verification Budd is asking the industry for, the safest way to review a generated screen is still the one that skips the write-up and opens the build.
