A study of 5,851 real developer sessions and 355,942 tool calls, led by Obada Kraishan and Kulsawasd Jitkajornwanich, finds that the summary a coding agent writes about its own work references roughly one action in eleven of what actually happened — and leans hardest on the original plan precisely when execution has drifted away from it. The episode walks through both headline numbers, the paper’s own validation check that failed outright, and the desktop tool developers are already building by hand to see the log the summary compressed away.
It also covers the honest limits of the study: the authors never tested whether thin summaries lead to worse code review outcomes, and a low-coverage report didn’t reliably predict a session that needed human intervention. Their own proposed fix is modest — show the self-report as one record among three, not as the record.