The accessibility check has been sitting in your pipeline for months, and it has passed every build. Nobody on the team has ever watched it go red; everyone reads the green tick as coverage, not as the narrow question it actually asks. A paper posted 8 September 2026, Mishra, Karthik, Bae and Yoon’s “Agentic Web Accessibility Auditing” , measured that question against 250 real page-criterion records and found that axe-core, the tool most teams gate their CI on, misses close to two-thirds of the violations actually present. Newer AI agents that click, tab and scroll through a page the way a person does catch far more — and are wrong nearly half the time they flag something. Neither number is comfortable, and the industry has built its whole accessibility gate on the one that sounds safer.
Never crying wolf was the whole design brief
Axe-core’s caution is not an accident of engineering; it is the stated goal. Deque, the company behind the tool, wrote in 2021 that “the axe-core rules library places a huge emphasis on not reporting false positives or erroneous issues” — because a checker that cries wolf gets removed from CI within a sprint. That trade shows up exactly where you’d expect in the new paper’s reference run: axe-core scores 0.90 precision against just 0.36 recall. The agentic “workers” that operate pages instead of parsing them land almost the mirror image, 0.86 recall at 0.56 precision. One tool tells you almost nothing that isn’t true. The other tells you most of what’s wrong, padded with noise you have to sort through yourself.
| Category | axe-core | Agentic auditors |
|---|---|---|
| Precision | 0.90 | 0.56 |
| Recall | 0.36 | 0.86 |
The clearest case is the keyboard trap — a control a keyboard-only user can enter but cannot leave, invisible unless something actually presses Tab. In the paper’s reference run, the agent workers recovered seven of eight Keyboard cases and both No Keyboard Trap cases; the uncued batch vision-language baseline recovered none of them, and axe-core missed every one too, because detecting a trap requires operating the page rather than reading it. A page can fail that way and still pass every automated check that has ever run against it.
The industry has built its whole accessibility gate on the score that sounds safest, not the one that catches the most problems.
Figma’s checker draws the same line one layer up
That same read-only shape now sits one layer earlier in the pipeline, inside the design file itself. Figma’s AI accessibility checker , in open beta, audits contrast ratios, text legibility, touch targets and colour blindness, and can propagate fixes across every instance of a design-system component. Figma’s own FAQ concedes the tool is “effective at catching quantifiable issues like contrast ratios and minimum sizing” but “less reliable for contextual issues that require human judgment, like whether alt text is meaningful or navigation is logical.” Keyboard navigation, focus order and focus traps appear nowhere on the product page — not flagged as a limitation, simply absent, which is precisely the criterion the agentic auditors recovered when both read-only baselines missed it.
For the frontend engineer or design-system maintainer who wired axe into CI months ago and considered the job done, that gap has a name and a location. The green tick certifies that no rule-detectable violation exists in the markup — not that a keyboard user can get out of the modal your team shipped last sprint. It matters more now that components are as likely to come from an agent as from a hand: an AI-generated modal can match every contrast and sizing rule your gate checks and still fail on the behavior your gate structurally cannot reach.
Both scores might be measuring past each other
The honest complication is that neither figure is as clean as it looks. Deque’s long-standing claim that automated testing covers 57% of issues was measured across 2,000-plus audits by volume of issues found, not by per-criterion recall on 250 records from eleven library platforms — a different yardstick, and Deque has argued directly that counting by WCAG criterion “downplays the effectiveness of automated testing.” The agents’ 0.56 precision may also understate their real accuracy: the paper’s own authors note that negative labels “were not collected through an independent pass/fail audit,” so some of what looks like a false alarm may be a real issue the original human auditors simply never logged.
None of the 250 records tested an AI-generated interface — every platform in the study was human-built. Read across to the pages coding agents are now shipping, though, WebAIM’s 2026 scan of a million home pages found average errors per page up 10.1% year over year, a rise it says “likely reflects” growing reliance on “automated or AI-assisted coding practices (‘vibe coding’)” — a correlational aside, not a measured cause, but one that points the same direction as the paper’s finding: the same instruction-following gap that keeps AI agents from tracking a design system’s own documented rules is not one a precision-tuned checker will ever be built to see. The gate was never wrong to trust rules. It was only ever built to read a page, not to use one.
