PNN2, a near-novice working through a mapping exercise, couldn’t get the ordinary click-and-drag controls to do what they wanted — and a prompt box sat open the entire time, right beside those controls, without ever being used. Asked why, they said: “I wasn’t sure how to prompt it. I wasn’t sure if it had to be in technical terms.” That moment is the finding in miniature of a new OOPSLA 2026 paper by Parker Ziegler, David Minh-Duy Cao, Justin Lubin and Sarah E. Chasins: across 2,946 edits made by 18 participants in their study, only 6.14% were typed as natural-language prompts. Every major design and coding tool right now is betting the opposite way, wiring a chat panel next to the canvas on the assumption that describing an edit will replace performing it — but given both options in the same interface, the people in this study overwhelmingly kept clicking.
What the mouse kept and what the prompt actually won
The researchers built cartokitDM+NL, a map-editing tool where both a mouse and a prompt box modify the same underlying program, so a typed instruction could never produce a broken result the way a bolted-on AI feature sometimes can — about as fair a test as this comparison gets. The prompt still lost badly, but not everywhere. Participants reached for it in two specific situations: batch edits that would have cost six to nine separate clicks by hand, and “moonshot” attempts at things they assumed the graphical interface simply couldn’t do. One participant, setting the break points on a map’s color scale, gave up on the sliders and instead asked for breaks at “integer values in 10 percentage-point increments moving away from zero” — a single sentence standing in for what would otherwise have been six to nine manual interactions. That’s the prompt earning its keep as a shortcut over the interface, not a substitute for it.
Given a mouse and a prompt box in the same editor, people typed prompts for only 6% of their edits.
The honest complication in shipped-product data
The steelman here is real, and it arrived just eleven days before this preprint. In her account of internal A/B tests , Amplitude’s Amy Lin reports a chat-first home page raised engagement 55% with under 3% of users reverting, an always-open AI sidebar beating a corner icon by 16.6%, and messages sent up 13x since December 2025. Separately, JetBrains’ 15,000-developer survey shows the market moving toward prompt-only tools, not away from them: Claude Code, a command-line tool with no direct-manipulation surface, grew from 18% adoption in January 2026 to roughly 39% by mid-year, overtaking GUI-native incumbents Copilot and Cursor, both declining over the same period — a pattern Pipeline has also traced through how those Claude Code teams actually coordinate . Neither result contradicts the OOPSLA study once read past its headline: Lin’s own conclusion is that “prompts only work when they’re anchored to user intentions,” and the paper’s authors, working from one 18-person study of a single tool, decline to argue either paradigm is better. Claude Code wins in an editor with no mouse to compete against; the prompt box in cartokitDM+NL wins only the fraction of jobs a mouse handles badly.
What this means for the products actually shipping chat panels
Neither study measured Figma, v0 or Lovable — the researchers tested one research tool across two short tasks, and JetBrains counted which coding tools professionals adopted, not how those tools’ internal panels get used. Applying the cartokit finding to design tools is a read-across, not a measurement — but Figma’s own release notes show exactly the bet this study should give pause: a late-August update giving its AI agent panel a separate floating window, followed on 1 September by MCP-connected generative plugins with their own code viewer. If your product’s canvas already has good direct-manipulation controls, a chat panel bolted beside them is not competing on equal footing for most of a user’s day — it is competing for the narrow slice of edits that are repetitive or that users have already decided, rightly or wrongly, the canvas can’t do. The same mismatch shows up one layer down in Figma Make , where properties-panel sliders quietly route every adjustment back through a prompt to the model rather than editing the file directly — treating the panel as decoration over a chat interface that was never optional to begin with.
That reframing matters most for the designer or engineer deciding, right now, whether their product’s next AI affordance should be a chat panel: the panel will sit closed through most of a user’s working session, and it will be opened for roughly two jobs — batching what the mouse does slowly, and reaching for what someone assumes the mouse can’t do. A team measuring an AI panel by total messages sent is measuring engagement with the feature, not whether it replaced anything; a team measuring it against those two specific jobs is measuring whether it earned its place on the screen. PNN2 never found out what to type. The tool never told them, and the study suggests most of their fellow participants never needed to ask.
