Two Kinds of Code Review
Comments on “Two Kinds of Code Review”
Claim-checked. Scored by all three judges, and its checkable claims were compared with the sources it cites. How far stories are checked
Hayden Bleasel, an engineer at OpenAI, argues that code review is two jobs: objective review (does it work) and subjective review (is it the right change, in scope and direction). He says agents now do the first better, and people should skim file lists and approach. He cites OpenAI evals where Codex with written rules caught 98% of targeted findings versus 58% without. Several cited review studies and Faros figures could not be verified.
Why it ranked
No. 7 of 111 with a score of 0.72. The panel rated it highest on practical relevance (0.86) and audience breadth (0.83), lowest on significance (0.63). Placed No. 7, No. 11 by score alone: close to “One reviewer per layer: splitting code review by the way code is read”, No. 23 on Wednesday 28 October 2026 (−0.011). The placement terms move stories; they are not part of the score.
- AI judgeJudge 2claude-haiku-5-5
- Practical essay on splitting code review into objective and subjective parts for agent-written code, with cited studies and industry examples; substantive and actionable but opinion-driven with some unverified claims.
- AI judgeJudge 3gpt-5.6-luna
- A well-supported, highly actionable analysis of how AI changes code review, distinguishing objective defect detection from human judgment about scope and direction. Its main weakness is reliance on cited claims without accessible evidence and some broad extrapolation.
Each judge wrote its line after answering the rubric, with the site and the author still hidden. Every answer behind this is in the table under Score.
Score
0.72total under rubric v2.3
- Significance0.63
- Reader interest0.83
- Audience breadth0.83
- Practical relevance0.86
- Substantiated0.65
- Technical depth0.64
- New information0.76
- Counts against: Promotional0.10
- Counts against: Overclaim0.28
Each judge’s answer per question (0–1), their mean and spread, and what the question adds to the score (weight × mean).
| Question | Judge 1 | Judge 2 | Judge 3 | Mean | Spread | Weight | Contribution (weight times mean) |
|---|---|---|---|---|---|---|---|
| SignificanceHow much does this change what people in tech know or can do? | 0.73 | 0.45 | 0.72 | 0.63 | 0.28 | +0.24 | +0.152 |
| Reader interestWould a well-informed senior engineer or tech founder want to read this today, and likely pass it on? | 0.78 | 0.80 | 0.90 | 0.83 | 0.12 | +0.26 | +0.215 |
| Audience breadthHow many people who build technology does it concern? | 0.90 | 0.80 | 0.78 | 0.83 | 0.12 | +0.13 | +0.108 |
| Practical relevanceHow useful is it to someone who builds software or tech products? | 0.88 | 0.85 | 0.86 | 0.86 | 0.03 | +0.22 | +0.190 |
| SubstantiatedHow well is it backed? Consider whether it is the primary source, and whether its main claims come with data, code, a demo, a reproducible method or citations. | 0.62 | 0.60 | 0.72 | 0.65 | 0.12 | +0.05 | +0.032 |
| Technical depthHow technically substantive is it? | 0.69 | 0.55 | 0.67 | 0.64 | 0.14 | +0.05 | +0.032 |
| New informationDoes it contain information absent from the stories covered in the last 7 days (list provided)? | 0.85 | 0.60 | 0.82 | 0.76 | 0.25 | +0.05 | +0.038 |
| PromotionalIs it mainly marketing with little substance (a press release, ad or sponsored post with no real information)? A launch post that carries real news is not promotional. | 0.07 | 0.15 | 0.08 | 0.10 | 0.08 | −0.19 | −0.019 |
| OverclaimDo the headline or conclusions claim more than the body supports? | 0.22 | 0.30 | 0.32 | 0.28 | 0.10 | −0.11 | −0.031 |
| Sum of contributions | +0.717 | ||||||
| Claim-check penalty | 0.000 | ||||||
| Total, as published | 0.717 | ||||||
Spread is the highest minus the lowest judge answer. marks a spread above 0.40: the judges disagree, and the story is marked contested.
The badge for this ranking, for the author's page or anyone's: one line of HTML, and it links back here.
<a href="https://rubric.hrih.si/editions/2026-10-31/7/"><img src="https://rubric.hrih.si/badge/2026-10-31/7.svg" alt="Ranked #7 on Rubric, 31 Oct 2026" width="238" height="28"></a>Claim check
Claim check: 4 matches source · 4 couldn’t check. Each claim is compared with the source the article itself cites; a flag is published only when both checkers agree.
Matches sourceNumber
Anthropic merged about 2,500 code changes a day in May, and over 30,000 a day by September.
The cited source says the same thing.
The article says “In May, Anthropic merged about 2,500 code changes a day. By September it was over 30,000”
The source says “In May 2026, Anthropic averaged around 2,500 code changes merged per day; as of September, we merged over 30,000 per day”
Source cited by the article: anthropic.com/research/redesigning-source-control
Couldn’t checkNumber
Mäntylä and Lassenius found that about three in four findings in code reviews did not change how the code behaved.
The source was unreachable, paywalled or too long, or the two checkers disagreed.
The article says “about three in four findings didn’t change how the code behaved”
The source says No passage could be compared.
Source cited by the article: doi.org/10.1109/TSE.2008.71
Couldn’t checkNumber
Bacchelli and Bird found at Microsoft that defects made up only 14% of review comments.
The source was unreachable, paywalled or too long, or the two checkers disagreed.
The article says “defects made up only 14% of review comments”
The source says No passage could be compared.
Source cited by the article: microsoft.com/en-us/research
Matches sourceNumber
Authors change their code after about half of the comments from Codex's automated review of internal OpenAI PRs.
The cited source says the same thing.
The article says “authors change their code after about half of its comments”
The source says “When the reviewer leaves a comment, authors address it with a code change in 52.7% of cases (often assisted by Codex”
Source cited by the article: alignment.openai.com/scaling-code-verification
Couldn’t checkNumber
Faros found that as teams leaned on AI, median time in review rose 441% and 31% more PRs were merged with no review.
The source was unreachable, paywalled or too long, or the two checkers disagreed.
The article says “median time in review went up 441%, and 31% more PRs were merged with no review at all”
The source says No passage could be compared.
Source cited by the article: pages.faros.ai/hubfs/AI_Engineering_Report_2026_The_Acceleration_Whiplash_Faros.pdf
Matches sourceNumber
An OpenAI team using GPT-6 to clean up its codebase merged more than 1,400 PRs and deleted over 800K lines, though some proposals added more complexity than they were worth.
The cited source says the same thing.
The article says “They merged more than 1,400 PRs and deleted over 800K lines, but some of the proposals added more complexity than they were worth”
The source says “we merged more than 1,400 pull requests, deleting over 800k lines of code”
Source cited by the article: developers.openai.com/blog/maintaining-software-with-the-gpt-6-family
Matches sourceBenchmark result
In OpenAI's evals, Codex review with written rules caught 98% of the findings the rules targeted, versus 58% without them.
The cited source says the same thing.
The article says “review with those rules caught 98% of the findings they were written for, compared with 58% without them”
The source says “In the primary suite, rule-guided variants recovered 98% of the required custom findings, compared with 58.3% in the baseline control”
Source cited by the article: developers.openai.com/blog/custom-code-review-rules-for-codex
Couldn’t checkNumber
In a study of 33K agent PRs, the unmerged ones were larger and touched more files, and a manual read of 600 found features nobody asked for.
The source was unreachable, paywalled or too long, or the two checkers disagreed.
The article says “the ones that didn’t get merged were larger and touched more files. A manual read of 600 of them turned up features nobody asked for”
The source says No passage could be compared.
Source cited by the article: arxiv.org/abs/2601.15195
Also covered by
No other coverage was clustered with this story.
Elsewhere
Elsewhere: HN 2 pts, not on front page
Recorded for context only, checked 31 Oct 2026, 05:46 UTC. It played no part in the ranking.