Why AI agents struggle with chip design (RTL) debugging, and what helps
Comments on “Why AI agents struggle with chip design (RTL) debugging, and what helps”
Claim-checked. Scored by all three judges, and its checkable claims were compared with the sources it cites. How far stories are checked
Booley's author, boldaxolotl, ran a case study with Claude Code on a reintroduced forwarding bug in the VeeR EL2 RISC-V core inside Caliptra. The agent found the faulty line within about five minutes either way, but reproducing the bug took far longer. Adding waveform search, blocking simulation calls and file-based build configuration cut mean time to a verified fix from 55.8 to 21.3 minutes over ten trials each. The author built the tool and ran the test, and the study covers one bug, one design and one model.
Why it ranked
No. 18 of 107 with a score of 0.62. The panel rated it highest on technical depth (0.92) and new information (0.91), lowest on audience breadth (0.44). Placed No. 18, No. 34 by score alone: the placement terms moved other stories past it.
- AI judgeJudge 2claude-haiku-5-5
- Substantive, well-evidenced experiment on AI agents debugging RTL with controlled trials and reproducible setup; strong technical depth, niche-to-moderate audience, promotional elements from the authors' own tool.
- AI judgeJudge 3gpt-5.6-luna
- A reproducible, technically detailed case study measures why general coding agents lose time debugging RTL and shows concrete harness improvements. Its evidence is strong but limited to one bug, design, model, and preconfigured project.
Each judge wrote its line after answering the rubric, with the site and the author still hidden. Every answer behind this is in the table under Score.
Score
0.62total under rubric v2.3
- Significance0.52
- Reader interest0.74
- Audience breadth0.44
- Practical relevance0.74
- Substantiated0.90
- Technical depth0.92
- New information0.91
- Counts against: Promotional0.16
- Counts against: Overclaim0.19
Each judge’s answer per question (0–1), their mean and spread, and what the question adds to the score (weight × mean).
| Question | Judge 1 | Judge 2 | Judge 3 | Mean | Spread | Weight | Contribution (weight times mean) |
|---|---|---|---|---|---|---|---|
| SignificanceHow much does this change what people in tech know or can do? | 0.54 | 0.35 | 0.68 | 0.52 | 0.33 | +0.24 | +0.125 |
| Reader interestWould a well-informed senior engineer or tech founder want to read this today, and likely pass it on? | 0.67 | 0.72 | 0.84 | 0.74 | 0.17 | +0.26 | +0.193 |
| Audience breadthHow many people who build technology does it concern? | 0.50 | 0.40 | 0.42 | 0.44 | 0.10 | +0.13 | +0.057 |
| Practical relevanceHow useful is it to someone who builds software or tech products? | 0.73 | 0.70 | 0.79 | 0.74 | 0.09 | +0.22 | +0.163 |
| SubstantiatedHow well is it backed? Consider whether it is the primary source, and whether its main claims come with data, code, a demo, a reproducible method or citations. | 0.99 | 0.80 | 0.91 | 0.90 | 0.19 | +0.05 | +0.045 |
| Technical depthHow technically substantive is it? | 1.00 | 0.85 | 0.90 | 0.92 | 0.15 | +0.05 | +0.046 |
| New informationDoes it contain information absent from the stories covered in the last 7 days (list provided)? | 0.88 | 0.90 | 0.96 | 0.91 | 0.08 | +0.05 | +0.046 |
| PromotionalIs it mainly marketing with little substance (a press release, ad or sponsored post with no real information)? A launch post that carries real news is not promotional. | 0.06 | 0.35 | 0.08 | 0.16 | 0.29 | −0.19 | −0.031 |
| OverclaimDo the headline or conclusions claim more than the body supports? | 0.20 | 0.30 | 0.08 | 0.19 | 0.22 | −0.11 | −0.021 |
| Sum of contributions | +0.623 | ||||||
| Claim-check penalty | 0.000 | ||||||
| Total, as published | 0.623 | ||||||
Spread is the highest minus the lowest judge answer. marks a spread above 0.40: the judges disagree, and the story is marked contested.
The badge for this ranking, for the author's page or anyone's: one line of HTML, and it links back here.
<a href="https://rubric.hrih.si/editions/2026-11-03/18/"><img src="https://rubric.hrih.si/badge/2026-11-03/18.svg" alt="Ranked #18 on Rubric, 3 Nov 2026" width="238" height="28"></a>Claim check
Claim check: 1 matches source · 6 no source given · 1 couldn’t check. Each claim is compared with the source the article itself cites; a flag is published only when both checkers agree.
No source givenNumber
With tools aimed at the three problems, the same agent's mean time to a verified fix fell from 56 minutes to 21 minutes.
The article gives no link or citation for it.
The article says “With tools aimed at all three, the same agent's mean time to a verified fix fell from 56 minutes to 21”
The source says No source to compare with.
No source givenNumber
Mean completion time was 55.8 minutes with Claude Code alone and 21.3 minutes with Booley; medians were 53.3 and 21.4 minutes.
The article gives no link or citation for it.
The article says “Mean completion time was 55.8 minutes with Claude Code alone and 21.3 minutes with Booley (Figure 1); the medians were 53.3 and 21.4 minutes”
The source says No source to compare with.
No source givenNumber
Average cost per trial at Sonnet 5.5 API prices was $2.33 for Claude Code alone and $1.98 with Booley.
The article gives no link or citation for it.
The article says “Average cost per trial at Sonnet 5.5 API prices was $2.33 and $1.98”
The source says No source to compare with.
Matches sourceQuote
The upstream VeeR EL2 release notes describe the fix as "Store data collision with DCCM DMA error when address is 0x0".
The cited source says the same thing.
The article says “Store data collision with DCCM DMA error when address is 0x0”
The source says “Store data collision with DCCM DMA error when address is 0x0”
Source cited by the article: github.com/chipsalliance/Cores-VeeR-EL2/blob/fb3354352be70781450ea70963c44d4b8f88ae9b/release-notes.md
Couldn’t checkVersion
The bug was a historical VeeR EL2 bug, reintroduced by reverting an upstream fix from the VeeR EL2 1.4 release.
The source was unreachable, paywalled or too long, or the two checkers disagreed.
The article says “reintroduced it by reverting an upstream fix from the VeeR EL2 1.4 release”
The source says No passage could be compared.
Source cited by the article: github.com/chipsalliance/Cores-VeeR-EL2/commit/fb3354352be70781450ea70963c44d4b8f88ae9b
No source givenNumber
With Booley, the agent spent 9.4 minutes less on average observing signals across all ten pairs.
The article gives no link or citation for it.
The article says “Across all ten pairs, it spent 9.4 minutes less on average observing signals with Booley”
The source says No source to compare with.
No source givenNumber
In trial b10, the plain-shell agent spent 30.7 of its 60 minutes asleep after jobs had finished, after a build failure hidden by '; tail' reporting exit code 0.
The article gives no link or citation for it.
The article says “spending 30.7 of its 60 minutes asleep after jobs had finished”
The source says No source to compare with.
No source givenCausal conclusion
The harness did not solve the microarchitectural reasoning, but it lowered the cost of each experiment enough to cut total time by more than half.
The article gives no link or citation for it.
The article says “It changed the cost of each experiment the agent ran on the way there, and that was enough to cut the total time by more than half”
The source says No source to compare with.
Also covered by
No other coverage was clustered with this story.
Elsewhere
Elsewhere: HN 1 pts, not on front page
Recorded for context only, checked 3 Nov 2026, 05:47 UTC. It played no part in the ranking.