More thinking isn't (always) better
Comments on “More thinking isn't (always) better”
Claim-checked. Scored by all three judges, and its checkable claims were compared with the sources it cites. How far stories are checked
A benchmark of five coding agents at up to three effort levels, on 8 tasks from five codebases, found that high effort roughly doubled diff size (mean 126 to 281 lines on solved runs) but raised solve rates for only two models, Opus 5.5 and GPT-6.1 Sol. Sonnet 5.5 and Fable 5.1 solved fewer tasks at high than at low, and Opus did worse at max than at high on the hardest bug. The author's figures are self-reported, cover a small task set, and suggest setting defaults per model.
Why it ranked
No. 12 of 111 with a score of 0.67. The panel rated it highest on practical relevance (0.87) and new information (0.86), lowest on significance (0.53). Placed No. 12, No. 17 by score alone: the placement terms moved other stories past it.
- AI judgeJudge 2claude-haiku-5-5
- Self-run benchmark of coding-agent effort levels with concrete solve rates, costs and diff sizes; practical for builders, modest significance, small sample size limits strength of conclusions.
- AI judgeJudge 3gpt-5.6-luna
- A practical, original measurement of coding-agent effort levels across models and repositories, with task outcomes, cost, time, and diff-size data. Small sample size limits generality; the consulting invitation is minor promotion.
Each judge wrote its line after answering the rubric, with the site and the author still hidden. Every answer behind this is in the table under Score.
Score
0.67total under rubric v2.3
- Significance0.53
- Reader interest0.78
- Audience breadth0.69
- Practical relevance0.87
- Substantiated0.76
- Technical depth0.67
- New information0.86
- Counts against: Promotional0.15
- Counts against: Overclaim0.22
Each judge’s answer per question (0–1), their mean and spread, and what the question adds to the score (weight × mean).
| Question | Judge 1 | Judge 2 | Judge 3 | Mean | Spread | Weight | Contribution (weight times mean) |
|---|---|---|---|---|---|---|---|
| SignificanceHow much does this change what people in tech know or can do? | 0.61 | 0.35 | 0.63 | 0.53 | 0.28 | +0.24 | +0.127 |
| Reader interestWould a well-informed senior engineer or tech founder want to read this today, and likely pass it on? | 0.73 | 0.70 | 0.90 | 0.78 | 0.20 | +0.26 | +0.202 |
| Audience breadthHow many people who build technology does it concern? | 0.74 | 0.60 | 0.72 | 0.69 | 0.14 | +0.13 | +0.089 |
| Practical relevanceHow useful is it to someone who builds software or tech products? | 0.94 | 0.80 | 0.86 | 0.87 | 0.14 | +0.22 | +0.191 |
| SubstantiatedHow well is it backed? Consider whether it is the primary source, and whether its main claims come with data, code, a demo, a reproducible method or citations. | 0.90 | 0.60 | 0.78 | 0.76 | 0.30 | +0.05 | +0.038 |
| Technical depthHow technically substantive is it? | 0.74 | 0.60 | 0.66 | 0.67 | 0.14 | +0.05 | +0.033 |
| New informationDoes it contain information absent from the stories covered in the last 7 days (list provided)? | 0.83 | 0.85 | 0.90 | 0.86 | 0.07 | +0.05 | +0.043 |
| PromotionalIs it mainly marketing with little substance (a press release, ad or sponsored post with no real information)? A launch post that carries real news is not promotional. | 0.08 | 0.20 | 0.18 | 0.15 | 0.12 | −0.19 | −0.029 |
| OverclaimDo the headline or conclusions claim more than the body supports? | 0.24 | 0.35 | 0.08 | 0.22 | 0.27 | −0.11 | −0.025 |
| Sum of contributions | +0.670 | ||||||
| Claim-check penalty | 0.000 | ||||||
| Total, as published | 0.670 | ||||||
Spread is the highest minus the lowest judge answer. marks a spread above 0.40: the judges disagree, and the story is marked contested.
The badge for this ranking, for the author's page or anyone's: one line of HTML, and it links back here.
<a href="https://rubric.hrih.si/editions/2026-10-31/12/"><img src="https://rubric.hrih.si/badge/2026-10-31/12.svg" alt="Ranked #12 on Rubric, 31 Oct 2026" width="245" height="28"></a>Claim check
Claim check: 2 matches source · 6 no source given. Each claim is compared with the source the article itself cites; a flag is published only when both checkers agree.
No source givenNumber
Turning coding agents up to high effort roughly doubled how much code they changed, and only 2 of 5 models solved more tasks with it.
The article gives no link or citation for it.
The article says “Turning coding agents up to high effort roughly doubled how much code they changed, and only 2 of 5 models solved more tasks with it”
The source says No source to compare with.
No source givenBenchmark result
Opus 5.5 at high effort was the only setting to fix the hardest bug every time (3 of 3, versus 1 of 3 at medium and max, and 0 for every other model).
The article gives no link or citation for it.
The article says “Opus at high was the only setting to fix the hardest bug every time (3 of 3, against 1 of 3 at medium and max, and 0 for every other…”
The source says No source to compare with.
No source givenBenchmark result
At each model's Claude Code default, Opus 5.5 (medium) beat Fable 5.1 (high) on solve rate, cost and time: 88%, $1.07 per run and 3m10s versus 75%, $3.56 and 6m28s.
The article gives no link or citation for it.
The article says “At each model’s Claude Code default, Opus 5.5 beat Fable 5.1 on solves, cost, and time”
The source says No source to compare with.
Matches sourceQuote
Anthropic's write-up on effort found that effort doesn't rescue a model that picks the wrong approach.
The cited source says the same thing.
The article says “Anthropic’s own write-up on effort found that effort doesn’t rescue a model that picks the wrong approach”
The source says “Overall, increasing effort tends to reduce failures due to missing edgecases (purple blocks), but does not fix when the model has the wrong approach (blue blocks”
Source cited by the article: claude.dev/blog/spending-your-effort
No source givenNumber
High effort levels make agent diffs 2.2x longer; for runs with a working solution the mean diff goes from 126 lines at low to 281 at high.
The article gives no link or citation for it.
The article says “For runs with a working solution the mean diff goes from 126 lines at low to 281 at high”
The source says No source to compare with.
No source givenBenchmark result
Opus 5.5 did worse at max effort than at high on the hardest bug, in three runs.
The article gives no link or citation for it.
The article says “The result was that Opus did worse at max than it did at high”
The source says No source to compare with.
Matches sourceCausal conclusion
Claude Code's docs warn about overthinking at max effort; Opus rewrote far more code at max than at high (a median of 664 lines versus 140).
The cited source says the same thing.
The article says “Claude Code’s own docs warn about at max: it rewrote far more code than at high (a median of 664 lines against 140”
The source says “max may show diminishing returns and is prone to overthinking, so test before adopting it broadly”
Source cited by the article: code.claude.com/docs/en/model-config
No source givenNumber
Sonnet 5.5 at low effort gets an 88% success rate at $0.31 per run, under a quarter of the $1.39 Opus at high charges.
The article gives no link or citation for it.
The article says “Gets an 88% success rate at $0.31 per run, under a quarter of the $1.39 Opus at high charges for more solves”
The source says No source to compare with.
Also covered by
No other coverage was clustered with this story.
Elsewhere
Elsewhere: HN 1 pts, not on front page
Recorded for context only, checked 31 Oct 2026, 05:46 UTC. It played no part in the ranking.