Victor Del Puerto, who runs a small engineering firm in Paraguay, had a shell-quoting trap written into his agent’s memory notes since 2 September. On 17 September the agent still wrote “transcribe with on CPU” instead of “transcribe with base on CPU”, because bash ran the backticked word as a command. By his account on HackerNoon
, it happened twice more within two hours. That is the bargain many developers now make with Claude Code and its peers: write the mistake down so it won’t recur. But the agent rarely opens the notes, and the one fix that held was a hard block that also stopped mostly harmless commands.
The note was there; the agent didn’t open it
Anthropic is candid about this. Its memory documentation says Claude treats CLAUDE.md files “as context, not enforced configuration”, and that to block an action regardless of what Claude decides, you should use a PreToolUse hook instead.
Del Puerto measured the gap. In his write-up , over two months he logged 699 firings of a hook that reminded the agent about relevant memory notes. A model judge found a relevant note in 471 of them. Afterwards the agent opened it in 111 cases, or 23.6%, which he describes as an upper bound. “The hook delivered its reminder every time,” he writes. The fscars README , his own project, restates the same figure, though with different denominators. Read cautiously, the reminder worked and the reading mostly didn’t: at most about a quarter of the time.
Here is where the finding reaches past what it measured. Claude Code’s auto-memory loads only the first 200 lines or 25KB of MEMORY.md at session start and, per the docs, keeps that file concise by “moving detailed notes into separate topic files.” Del Puerto measured his own reminder-to-note pointers, not Anthropic’s system. Read across, though, the shape matches: a detailed correction applies only if the agent decides to open the topic file, and that is the step his logs show failing most of the time.
Memory is a library, and libraries decay
The habit is spreading regardless. In a 9 October study , Nielsen Norman Group found Claude power users put more effort into engineering context libraries than into writing prompts. One kept over 2,700 context files. Files went stale or missing; one participant called it “system decay”. Another found a governing global file he did not know existed. The study is qualitative, and its sample size is not reported.
The pattern isn’t confined to one practitioner either. Academic work points the same way. A June paper on TRACE found that on tasks drawn from anonymized real-user friction, Mem0 memory still leaves 57.5% of applicable preference checks violated. Compiling user corrections into runtime checks that must pass before a task completes cut held-out violations from 100.0% to 2.0% on one out-of-distribution benchmark. The evaluation uses simulated users, so treat the numbers as direction, not rate.
Memory stores information, not behavior: a correction that must hold has to live in a check that runs outside the text the model reads.
Put the two together and Pipeline’s reading is this: memory stores information, not behavior. Neither study measured everyday CLAUDE.md practice; one is a single workspace, the other a simulation. The conclusion that corrections which must hold belong outside the model’s reading list is an inference, one that Anthropic’s stated design intent supports.
Enforcement works, and charges for it
Del Puerto’s blocking hook, which rejected heredoc writes to .py files, cut those heredoc writes from 6.7% to 0.8% of Bash calls across 102,298 calls, and the “unterminated string literal” errors they caused from 47 to 0. In the docs’ hooks reference , the mechanism is simple: exit code 2 blocks the action, and exit code 1 does not. As he puts it, “a scar is code that runs on every tool call, so it fails like code.”
That also means it fails like code in the bad ways. Of the commands the hook blocked, 92% would have passed intact. The share of sessions that tried the pattern at least once did not fall. His estimated benefit was only four to six prevented errors over 24 days. And he says such checks cannot catch new kinds of mistakes or judgment calls.
Which lines in your CLAUDE.md are rules?
None of this makes instructions worthless. ETH Zurich’s AGENTbench work found AGENTS.md instructions measurably change what coding agents do. And Del Puerto’s data is one operator’s observational logs; he notes that no controlled comparison of note, context and scar delivery has been run.
The practical shift is for the developer who has answered every repeated mistake by adding one more line. Now you sort those lines. A few are rules that must hold. Those become hook scripts that block the action, tested and retired like any other code. The rest stay as advice the agent may never open.
