A benchmark called Design Theater scored 120 interfaces generated by tools like ChatGPT, Claude, Firebase Studio, Vercel v0 and Bolt against the written rationales those same tools produced for their own design choices. More than a quarter of the explanations described something the code didn’t actually do, and the gap nearly doubles when you isolate functional requirements like keyboard navigation and error handling. Fidelity holds up fine on anything a screenshot would catch — styling, structure — and falls apart on anything that requires pressing a key.
The hosts trace that pattern through design consultant Andy Budd’s warning that “plausible design is dangerous,” through Nielsen Norman Group’s Raluca Budiu on why one good AI output is an example and not an evaluation, and through the benchmark’s own limitations section, which constrained every tool to raw HTML and CSS and admits that choice alone may have depressed the functional scores. The full article, with sources, is at pipelinemag.ai.