Google: Old TODO comments send coding agents chasing bugs fixed years ago
Comments on “Google: Old TODO comments send coding agents chasing bugs fixed years ago”
Claim-checked. Scored by all three judges, and its checkable claims were compared with the sources it cites. How far stories are checked
Google engineer Sam Saccone reports that stale TODO(bug/...) comments distract coding agents in Google's internal traces. About 13% of file-reading sessions encountered one; agents acted on 6% of encounters, mostly unprompted. Of those detours, roughly 41% chased closed tickets. Regression estimates put the cost at 70% more steps, 77% more tokens and 105% more wait time per session, though these figures are unverified and the causal framing rests on observational data. Google's cleanup agent has removed thousands of obsolete references, with over 95% reviewer acceptance.
Why it ranked
No. 9 of 51 with a score of 0.65. The panel rated it highest on new information (0.89) and practical relevance (0.77), lowest on significance (0.60). Placed No. 9, No. 10 by score alone: the placement terms moved other stories past it.
- AI judgeJudge 2claude-haiku-5-5
- Practical, data-backed finding on coding agents being distracted by stale TODO references, with telemetry metrics and a cleanup approach; largely new, though claims rest on internal data without independent verification.
- AI judgeJudge 3gpt-5.6-luna
- A data-rich empirical study of coding-agent behavior offers actionable engineering guidance and a novel maintenance concern. It is relevant and technically substantive, though methodology and evidence are summarized rather than reproducibly documented.
Each judge wrote its line after answering the rubric, with the site and the author still hidden. Every answer behind this is in the table under Score.
Score
0.65total under rubric v2.3
- Significance0.60
- Reader interest0.77
- Audience breadth0.68
- Practical relevance0.77
- Substantiated0.72
- Technical depth0.66
- New information0.89
- Counts against: Promotional0.17
- Counts against: Overclaim0.31
Each judge’s answer per question (0–1), their mean and spread, and what the question adds to the score (weight × mean).
| Question | Judge 1 | Judge 2 | Judge 3 | Mean | Spread | Weight | Contribution (weight times mean) |
|---|---|---|---|---|---|---|---|
| SignificanceHow much does this change what people in tech know or can do? | 0.67 | 0.45 | 0.67 | 0.60 | 0.22 | +0.24 | +0.143 |
| Reader interestWould a well-informed senior engineer or tech founder want to read this today, and likely pass it on? | 0.73 | 0.70 | 0.88 | 0.77 | 0.18 | +0.26 | +0.200 |
| Audience breadthHow many people who build technology does it concern? | 0.76 | 0.60 | 0.67 | 0.68 | 0.16 | +0.13 | +0.088 |
| Practical relevanceHow useful is it to someone who builds software or tech products? | 0.80 | 0.70 | 0.82 | 0.77 | 0.12 | +0.22 | +0.170 |
| SubstantiatedHow well is it backed? Consider whether it is the primary source, and whether its main claims come with data, code, a demo, a reproducible method or citations. | 0.77 | 0.60 | 0.78 | 0.72 | 0.18 | +0.05 | +0.036 |
| Technical depthHow technically substantive is it? | 0.74 | 0.55 | 0.68 | 0.66 | 0.20 | +0.05 | +0.033 |
| New informationDoes it contain information absent from the stories covered in the last 7 days (list provided)? | 0.84 | 0.90 | 0.92 | 0.89 | 0.08 | +0.05 | +0.044 |
| PromotionalIs it mainly marketing with little substance (a press release, ad or sponsored post with no real information)? A launch post that carries real news is not promotional. | 0.18 | 0.15 | 0.18 | 0.17 | 0.03 | −0.19 | −0.032 |
| OverclaimDo the headline or conclusions claim more than the body supports? | 0.35 | 0.30 | 0.27 | 0.31 | 0.08 | −0.11 | −0.034 |
| Sum of contributions | +0.648 | ||||||
| Claim-check penalty | 0.000 | ||||||
| Total, as published | 0.648 | ||||||
Spread is the highest minus the lowest judge answer. marks a spread above 0.40: the judges disagree, and the story is marked contested.
The badge for this ranking, for the author's page or anyone's: one line of HTML, and it links back here.
<a href="https://rubric.hrih.si/editions/2026-11-08/9/"><img src="https://rubric.hrih.si/badge/2026-11-08/9.svg" alt="Ranked #9 on Rubric, 8 Nov 2026" width="231" height="28"></a>Claim check
Claim check: 8 no source given. Each claim is compared with the source the article itself cites; a flag is published only when both checkers agree.
No source givenNumber
13% of file-reading agent sessions exposed the agent to inline TODO(bug/...) or FIXME(bug/...) bug references.
The article gives no link or citation for it.
The article says “13% of sessions exposed the agent to inline TODO(bug/...) or FIXME(bug/...) bug references”
The source says No source to compare with.
No source givenNumber
Agents ignored the bug-reference comment in 94% of encounters; in 6% they took subsequent action targeting that bug ID, and 83% of those actions were unprompted.
The article gives no link or citation for it.
The article says “While agents ignored the comment in 94% of encounters, in 6% the agent took subsequent action targeting that bug ID”
The source says No source to compare with.
No source givenNumber
Among unprompted detours, over two-fifths chased already closed tickets: 32% fixed/verified bugs and 9% obsolete/won't-fix, with an additional 2% duplicate or restricted tickets.
The article gives no link or citation for it.
The article says “split between already fixed/verified bugs (32%) and obsolete/won't-fix dead ends (9%), with an additional 2% hitting duplicate or restricted tickets”
The source says No source to compare with.
No source givenNumber
In over half of unprompted detours the bug was still open, and the agent expanded its work, with 61% lookup only and 39% editing code.
The article gives no link or citation for it.
The article says “the agent expanded its work (61% lookup only; 39% edited code) to investigate or modify an issue the developer never asked to touch”
The source says No source to compare with.
No source givenNumber
Unprompted bug detours add 70% more steps, 77% more tokens and 105% more wait time per session, all statistically significant at p<0.0001, after controlling for several factors via fixed-effects regression.
The article gives no link or citation for it.
The article says “70% more steps - 77% more tokens - 105% more wait time”
The source says No source to compare with.
No source givenCausal conclusion
Unprompted bug detours cause the extra steps, tokens and wait time in agent sessions.
The article gives no link or citation for it.
The article says “we see that unprompted bug detours add substantial overhead (all stat sig p<0.0001”
The source says No source to compare with.
No source givenNumber
Google's TODO cleaner agent has removed thousands of obsolete references over the last 6 months, with over 95% acceptance from human reviewers.
The article gives no link or citation for it.
The article says “our todo cleaner has been able to remove thousands of actually obsolete references over the last 6 months exceeding 95% acceptance from human reviewers”
The source says No source to compare with.
No source givenDate
In one trajectory, a bug fixed in 2023 remained as a TODO in SurfaceControlTest.java and distracted the agent into investigating it.
The article gives no link or citation for it.
The article says “This bug, while fixed in 2023, remains lingering in the codebase leading to subsequent investigation distraction”
The source says No source to compare with.
Also covered by
No other coverage was clustered with this story.
Elsewhere
Elsewhere: HN 2 pts, not on front page
Recorded for context only, checked 8 Nov 2026, 05:46 UTC. It played no part in the ranking.