Tag

Benchmarks

4 articles

A bold vintage screen-print illustration in pale cream, deep pine green and rust orange of a large calibration dial gauge crossed by a socket wrench, set against a scoreboard-tile band and diagonal stripes.

Development

The SWE-Bench Leaderboard Can No Longer Tell Models Apart

Two September 2026 papers, days apart, find SWE-bench Verified can't statistically separate its top coding agents, and the harness deciding the score resets with every model swap.

A vintage screen-print illustration in parchment cream, deep teal-green and vermillion red of a fanned stack of tracing-paper sketch sheets crossed diagonally by an oversized red pencil, with scattered eraser-shaving triangles below.

Prototyping

Generative UI Tools Are Benchmarked on a Turn, Used in a Session

Two 2026 studies (EvoGenUI-Bench, Maru) find generative-UI sessions degrade because each fix silently undoes an earlier one — not because models misread later prompts.

A bold vintage screen-print illustration in cream, deep teal-green and burnt rust orange of a thick zigzag folding tape measure stretched diagonally across a row of rectangular screen panels of different sizes.

Design Engineering

AI Writes Responsive Code That Isn't Responsive

A 12 August 2026 benchmark found 68% of AI-generated webpages break across real browsers and devices, 1.7x the human baseline, while reading fine in a diff.

A bold vintage screen-print illustration in deep burgundy, warm gold and muted teal of a large theatrical drama mask split down the middle, one half a smooth painted face and the other broken into a blank tile grid.

Prototyping

Design Theater: The Gap Between an AI's Rationale and the Screen

A July 2026 benchmark found over a quarter of AI design tools' stated rationales don't match the interfaces they built, and the gap is worst on behavior, not looks.