We review 4x more AI code without reading every line
Comments on “We review 4x more AI code without reading every line”
Claim-checked. Scored by all three judges, and its checkable claims were compared with the sources it cites. How far stories are checked
Ankit Jain of Aviator says his team stopped reading every diff line and instead has humans review intent, acceptance criteria, invariants, decisions and evidence from agent-run preview environments. He reports that the same nine engineers merged 4× more PRs, median time to merge fell from 2.1 days to 3.4 hours, and reverts dropped 40%. He also describes mining 11,400 review comments into 64 approved invariants. These figures are self-reported by the vendor and have no independent verification, and they come from one team.
Why it ranked
No. 10 of 46 with a score of 0.64. The panel rated it highest on reader interest (0.83) and practical relevance (0.80), lowest on substantiated (0.59). Placed No. 10, No. 11 by score alone: the placement terms moved other stories past it.
- AI judgeJudge 2claude-haiku-5-5
- Company-authored claims about AI code review workflow with self-reported metrics (4x PRs, 40% fewer reverts) and little independent verification; practical and on-topic but promotional-leaning and overclaim-prone.
- AI judgeJudge 3gpt-5.6-luna
- A detailed, actionable engineering account of restructuring AI-assisted code review around intent, invariants, evidence, and decisions, with concrete workflow and self-reported metrics. Some promotional and generalization risk remains because evidence is from one company.
Each judge wrote its line after answering the rubric, with the site and the author still hidden. Every answer behind this is in the table under Score.
Score
0.64total under rubric v2.3
- Significance0.62
- Reader interest0.83
- Audience breadth0.69
- Practical relevance0.80
- Substantiated0.59
- Technical depth0.71
- New information0.77
- Counts against: Promotional0.27
- Counts against: Overclaim0.42
Each judge’s answer per question (0–1), their mean and spread, and what the question adds to the score (weight × mean).
| Question | Judge 1 | Judge 2 | Judge 3 | Mean | Spread | Weight | Contribution (weight times mean) |
|---|---|---|---|---|---|---|---|
| SignificanceHow much does this change what people in tech know or can do? | 0.70 | 0.50 | 0.67 | 0.62 | 0.20 | +0.24 | +0.150 |
| Reader interestWould a well-informed senior engineer or tech founder want to read this today, and likely pass it on? | 0.81 | 0.80 | 0.88 | 0.83 | 0.08 | +0.26 | +0.216 |
| Audience breadthHow many people who build technology does it concern? | 0.76 | 0.67 | 0.63 | 0.69 | 0.13 | +0.13 | +0.089 |
| Practical relevanceHow useful is it to someone who builds software or tech products? | 0.88 | 0.70 | 0.83 | 0.80 | 0.18 | +0.22 | +0.177 |
| SubstantiatedHow well is it backed? Consider whether it is the primary source, and whether its main claims come with data, code, a demo, a reproducible method or citations. | 0.72 | 0.40 | 0.65 | 0.59 | 0.32 | +0.05 | +0.029 |
| Technical depthHow technically substantive is it? | 0.81 | 0.60 | 0.72 | 0.71 | 0.21 | +0.05 | +0.035 |
| New informationDoes it contain information absent from the stories covered in the last 7 days (list provided)? | 0.78 | 0.60 | 0.92 | 0.77 | 0.32 | +0.05 | +0.038 |
| PromotionalIs it mainly marketing with little substance (a press release, ad or sponsored post with no real information)? A launch post that carries real news is not promotional. | 0.12 | 0.40 | 0.28 | 0.27 | 0.28 | −0.19 | −0.051 |
| OverclaimDo the headline or conclusions claim more than the body supports? | 0.31 | 0.60 | 0.34 | 0.42 | 0.29 | −0.11 | −0.046 |
| Sum of contributions | +0.638 | ||||||
| Claim-check penalty | 0.000 | ||||||
| Total, as published | 0.638 | ||||||
Spread is the highest minus the lowest judge answer. marks a spread above 0.40: the judges disagree, and the story is marked contested.
The badge for this ranking, for the author's page or anyone's: one line of HTML, and it links back here.
<a href="https://rubric.hrih.si/editions/2026-11-09/10/"><img src="https://rubric.hrih.si/badge/2026-11-09/10.svg" alt="Ranked #10 on Rubric, 9 Nov 2026" width="238" height="28"></a>Claim check
Claim check: 7 no source given · 1 couldn’t check. Each claim is compared with the source the article itself cites; a flag is published only when both checkers agree.
No source givenNumber
At Aviator, the same nine engineers merged 4× more pull requests after changing their review approach.
The article gives no link or citation for it.
The article says “that shift let the same nine engineers merge 4× more pull requests”
The source says No source to compare with.
Couldn’t checkNumber
Aviator cut median time to merge from 2.1 days to 3.4 hours.
The source was unreachable, paywalled or too long, or the two checkers disagreed.
The article says “cut median time to merge from 2.1 days to 3.4 hours”
The source says No passage could be compared.
Source cited by the article: thenewstack.io/ai-code-review-pipelines
No source givenNumber
Aviator's reverts dropped by 40%.
The article gives no link or citation for it.
The article says “drop reverts by 40%”
The source says No source to compare with.
No source givenNumber
Aviator's weekly merged PRs went from about 100 to more than 400, and the median AI-generated PR reached 640 lines.
The article gives no link or citation for it.
The article says “Weekly merged PRs went from about 100 to more than 400. Our median AI-generated PR reached 640 lines”
The source says No source to compare with.
No source givenNumber
In Microsoft's 2013 study by Bacchelli & Bird, only 14% of review comments were about defects.
The article gives no link or citation for it.
The article says “In Microsoft’s 2013 study (Bacchelli & Bird), only 14% of review comments were about defects”
The source says No source to compare with.
No source givenNumber
Aviator mined 11,400 review comments into 212 candidate rules and 64 approved invariants.
The article gives no link or citation for it.
The article says “11,400 comments became 212 candidate rules and 64 approved invariants”
The source says No source to compare with.
No source givenNumber
Nineteen of Aviator's invariants run deterministically as ast-grep rules in 0.3 seconds.
The article gives no link or citation for it.
The article says “Nineteen invariants run deterministically, in 0.3 seconds”
The source says No source to compare with.
No source givenNumber
Human review threads per PR at Aviator fell from 6.2 to 1.3.
The article gives no link or citation for it.
The article says “Human threads per PR: 6.2 → 1.3”
The source says No source to compare with.
Also covered by
No other coverage was clustered with this story.
Elsewhere
Elsewhere: HN 3 pts, not on front page
Recorded for context only, checked 9 Nov 2026, 05:46 UTC. It played no part in the ranking.