You run /init, skim the description of your own codebase that Claude Code or Codex hands back, commit the resulting AGENTS.md, and move on. Nothing since has told you whether an agent read a line of it, or whether any of it changed what the agent did. Two 2026 studies now answer that, and the file splits cleanly in half. ETH Zurich’s evaluation of AGENTS.md files
, tested against 138 AGENTbench instances across 12 repositories plus 300 SWE-bench Lite instances with four coding agents, found the instructions genuinely land — but the repository tour, the part almost every generated file leads with, does not help an agent find the right code any faster, and still adds more than 20% to the cost of every run.
The instructions land; the tour doesn’t
The split shows up in what the agents actually do. Tool use per benchmark instance ranges from well under 0.05 uses when the tool goes unmentioned — under 0.01 for a tool like uv — up to 1.6 to 2.5 once it’s named inside the context file: from almost never to routine, once it’s written down. The overview does the opposite of its job: present in 95 to 100% of LLM-generated files, it never measurably speeds up how fast an agent finds the file it needs. For the engineer maintaining a CLAUDE.md or AGENTS.md, the architecture tour they keep polishing is billed on every run and buys nothing the study can detect, while the one line telling the agent to run make test before committing is the part that actually moves it. The ETH Zurich team draws a direct conclusion from that split:
“we suggest omitting LLM-generated context files for the time being … and including only minimal requirements (e.g., specific tooling to use with this repository).”
What agents open when nobody is grading them
A second study, published 20 August 2026, checks the same claim against real usage instead of a benchmark. Across 557 agent sessions and 33,097 pull requests, instruction files and agents’ own working notes accounted for 60.5% of all documentation interactions logged, against 10.6% for classical technical documentation and just 1.3% for API references, according to an empirical study of how coding agents discover, read and write documentation . Agents reached for that material on their own, too — consultation was self-initiated 70.2% of the time, against 7.5% that followed a visible failure — less a last resort than a habit already formed.
| Category | Share of interactions (%) |
|---|---|
| Instruction files & notes | 60.5 |
| Technical documentation | 10.6 |
| API references | 1.3 |
Neither study set out to grade how a team should divide its writing time. What they measured was task success on scoped benchmark instances, and which document types agents opened inside logged sessions. Read across to the file every repo maintains for its own long life, that gap suggests the effort spent writing prose descriptions of architecture is effort spent on the half of the document nobody is shown to use, while the instructions earn their keep from the first commit.
The architecture tour billed on every agent run buys nothing the study can detect, while the one line telling it to run tests is what actually moves it.
The two studies disagree about what happens next
That reading is tidier than the evidence fully allows, and the two papers don’t even agree with each other on a behavior that matters. ETH Zurich found that context files trigger more testing, searching and file reading — 2.45 to 3.92 extra steps per run, explaining the added cost without any gain in success. The 20 August trace study finds close to the opposite: a documentation-read event in its logs is followed by less immediate testing, not more, with a lift of 0.23 and an adjusted odds ratio of 0.39. The same ETH paper also found that developer-written context files lifted task success by about 4 percentage points, while LLM-generated ones cost about 3 points — human-authored context isn’t worthless, just poorly approximated by a generated summary. Neither study, either, covers the long-lived proprietary codebase most teams actually maintain, where a careful overview might carry weight a scoped benchmark task can’t register. That gap is why the case for CI-run evals on design-system rules matters too — measuring whether an agent obeys a file counts more than measuring whether it reads well.
One line, earned by a mistake
One practitioner already inverted the default. Guillaume Moigneu, writing about ETH Zurich’s paper on Upsun’s developer blog , read the findings as confirming “something a few of us had a gut-feeling about,” and proposed starting from an empty AGENTS.md rather than a generated one — adding a line only when an agent repeats a mistake, with no description of the repository’s architecture anywhere in the file. Every rule in that version was earned by a failure someone watched happen — the property the benchmark rewards and the generated overview never had. It’s the same instinct behind treating a repo’s context file as brand material rather than documentation : specific beats complete.
The odd part is that the file was never really failing at what it does — it’s failing at what it was never good at. Skim the architecture tour /init wrote you and you’re reading prose an agent doesn’t need to find its way around; delete the line telling it to run make test first, and you’d feel it by the next commit. The rulebook was always doing the work. The tour was just the part that made the file look thorough.
