Generative UI Tools Are Benchmarked on a Turn, Used in a Session

You ask for one more change, and something you’d already fixed two prompts ago quietly comes back. The layout that finally looked right in turn two reverts under a later edit, and the preview still renders cleanly, so nothing on screen tells you it happened. Two studies published days apart measured that exact experience, not the demo prompt vendors optimize for: EvoGenUI-Bench , from New York University Shanghai, and Maru , from KAIST and UC San Diego. Generative-UI tools are benchmarked and sold on their first output, but both papers find the real damage builds in the turns nobody screenshots — not because the model misreads the fifth request, but because answering it quietly breaks what the second one already got right.

Turn pass rates hide the whole session

EvoGenUI-Bench ran eight models through 150 five-turn tasks — 750 turns across suites covering presentation, interaction, and grounding an interface in an external tool. The strongest model, Claude Opus 4.7, passed 74.9% of individual turns. Only 37.3% of its five-turn episodes finished clean — and that isn’t even the alarming figure: independent turns at a 74.9% pass rate would complete just 23.6% of five-turn runs (0.749 to the fifth power), so the actual result means failures aren’t simply piling up independently. What isolates the real regression is what the paper calls Adjacent Pass Retention — given a turn that already passed, how often it still passes after the next revision. Overall, 66.5% of the time; on tasks requiring external grounding, 52.4%. The paper’s own case study makes it concrete: in a “Drone PID Wind Disturbance Lab” task, a user changes a controller’s gain values at turn five, the interface updates the displayed gains correctly, and the simulation metrics depending on them stay frozen at their old numbers. The surface is right. The thing underneath it isn’t.

Persisting structure recovers what regenerating erodes

Maru, presented at UIST 2026, is smaller and more granular: 12 participants, 36 sessions, 220 generated interfaces on personal-planning tasks like a graduate-school visit or a picnic. In the baseline condition — a system that rebuilds the interface from the conversation every turn, the way most tools work today — approval collapsed from 71% in a session’s first half to 33% in its second. On one task, all six participants converged on an identical table layout — a system with only the raw conversation to reconstruct from keeps reaching for the safest, most generic shape. When Maru instead persisted the user’s information architecture — the partitions, hierarchy, order and vocabulary the interface was built from — rather than regenerating it each turn, approval recovered to 61%, and those same six participants produced four distinct layouts instead of one. If you refine prototypes by typing changes into Figma Make, v0 or Lovable, this is what sits under your worst afternoons: the tool reverting a detail you’d already fixed isn’t confusion about your new request, it’s failing to keep what you’d settled two prompts back.

Neither study measured a designer iterating a prototype inside a commercial tool: EvoGenUI-Bench scores browser-executed benchmark tasks; Maru studies personal tasks, not product design. Read across to the tools designers use daily, that suggests the same regression shows up wherever a system regenerates rather than persists — an extension this article is making, not one either paper measured, and neither vendor publishes session-level data to confirm.

The persistent-file rebuttal has real weight, and real limits

The strongest objection to that extension is structural. Both benchmarks test systems that rebuild a UI from the conversation each turn, while Figma Make and v0 increasingly edit a persistent file or codebase — arguably the exact structural persistence Maru shows fixes the collapse. If a tool’s file survives a prompt the way Maru’s information architecture does, its session curve may sit far closer to 61% than to 33%. Figma’s own guidance leans on that gap without closing it: it advises prompting in small increments — “the smaller the scope, the more detailed the LLM can be” — and when iteration still goes wrong, its fix isn’t more prompting but abandoning the session: “if you find yourself making too many adjustments, consider starting fresh with a new Figma Make file.” A tool with genuine memory of its own file shouldn’t need that rip cord as standing advice.

Maru’s persistence has bounds the paper admits to: rules carried across unrelated tasks raised rejection to 23% (versus 18% within-topic), and sessions with 194 or more accumulated rules saw approval fall to 33–50% — the collapsing baseline’s own range. Persistence is a mechanism, not a cure — both papers are evidence of a mechanism, not a verified rate for any shipping product. That caution matches the properties panel Figma Make, v0 and Lovable restored in 2026 : vendor docs show it still serializes an edit into a prompt for the model to rewrite rather than touching the file directly — direct manipulation on the surface, generation underneath, exactly the seam these studies price out.

That mismatch — grading the fifth prompt as if it arrived alone — echoes coding agents too, where requirements landing after work has already started roughly double the rework no matter how the request is phrased. A benchmark that scores only whether one turn’s output is correct will always look better than a designer’s real Tuesday, because the designer is the one who has to notice, three prompts later, that the thing they already fixed has come undone.