Building a Software Factory
Comments on “Building a Software Factory”
Claim-checked. Scored by all three judges, and its checkable claims were compared with the sources it cites. How far stories are checked
Runway, a fintech software company, describes handing each stage of software delivery to agents: writing to green CI, review, risk-based approval, browser QA, fixing main, and coordination. It reports that agents drove 38% of October merges, median first-approval wait fell from over two hours to about five minutes, and AI-approved merges were reverted about one in a thousand against one in two hundred for human-approved ones. The figures are self-reported from internal data without external verification, and the mix of low-risk changes skews the revert comparison.
Why it ranked
No. 7 of 118 with a score of 0.72. The panel rated it highest on new information (0.89) and reader interest (0.87), lowest on technical depth (0.66). Placed No. 7, where the score alone would also have put it: close to “Linear changed code review for AI-written code”, No. 19 on Tuesday 17 November 2026 (−0.007). The placement terms move stories; they are not part of the score.
- AI judgeJudge 2claude-haiku-5-5
- Detailed, self-reported engineering account of agent-driven software delivery with concrete metrics, though mostly company-internal and unverifiable; some promotional framing and hiring pitch at the end.
- AI judgeJudge 3gpt-5.6-luna
- A detailed, metrics-backed engineering account of delegating software delivery stages to agents, including review, approval, QA, rollback, coordination, and operational guardrails. It is highly actionable, though evidence is internal and conclusions remain company-specific.
Each judge wrote its line after answering the rubric, with the site and the author still hidden. Every answer behind this is in the table under Score.
Score
0.72total under rubric v2.3
- Significance0.71
- Reader interest0.87
- Audience breadth0.71
- Practical relevance0.81
- Substantiated0.79
- Technical depth0.66
- New information0.89
- Counts against: Promotional0.18
- Counts against: Overclaim0.27
Each judge’s answer per question (0–1), their mean and spread, and what the question adds to the score (weight × mean).
| Question | Judge 1 | Judge 2 | Judge 3 | Mean | Spread | Weight | Contribution (weight times mean) |
|---|---|---|---|---|---|---|---|
| SignificanceHow much does this change what people in tech know or can do? | 0.77 | 0.55 | 0.82 | 0.71 | 0.27 | +0.24 | +0.171 |
| Reader interestWould a well-informed senior engineer or tech founder want to read this today, and likely pass it on? | 0.81 | 0.85 | 0.94 | 0.87 | 0.13 | +0.26 | +0.225 |
| Audience breadthHow many people who build technology does it concern? | 0.55 | 0.80 | 0.78 | 0.71 | 0.25 | +0.13 | +0.092 |
| Practical relevanceHow useful is it to someone who builds software or tech products? | 0.83 | 0.70 | 0.88 | 0.81 | 0.18 | +0.22 | +0.177 |
| SubstantiatedHow well is it backed? Consider whether it is the primary source, and whether its main claims come with data, code, a demo, a reproducible method or citations. | 0.94 | 0.60 | 0.84 | 0.79 | 0.34 | +0.05 | +0.040 |
| Technical depthHow technically substantive is it? | 0.76 | 0.45 | 0.78 | 0.66 | 0.33 | +0.05 | +0.033 |
| New informationDoes it contain information absent from the stories covered in the last 7 days (list provided)? | 0.84 | 0.90 | 0.93 | 0.89 | 0.09 | +0.05 | +0.045 |
| PromotionalIs it mainly marketing with little substance (a press release, ad or sponsored post with no real information)? A launch post that carries real news is not promotional. | 0.11 | 0.35 | 0.08 | 0.18 | 0.27 | −0.19 | −0.034 |
| OverclaimDo the headline or conclusions claim more than the body supports? | 0.24 | 0.40 | 0.18 | 0.27 | 0.22 | −0.11 | −0.030 |
| Sum of contributions | +0.719 | ||||||
| Claim-check penalty | 0.000 | ||||||
| Total, as published | 0.719 | ||||||
Spread is the highest minus the lowest judge answer. marks a spread above 0.40: the judges disagree, and the story is marked contested.
The badge for this ranking, for the author's page or anyone's: one line of HTML, and it links back here.
<a href="https://rubric.hrih.si/editions/2026-11-21/7/"><img src="https://rubric.hrih.si/badge/2026-11-21/7.svg" alt="Ranked #7 on Rubric, 21 Nov 2026" width="238" height="28"></a>Claim check
Claim check: 8 no source given. Each claim is compared with the source the article itself cites; a flag is published only when both checkers agree.
No source givenNumber
Cloud agents drove 38% of the pull requests merged in October, up from 0.1% in June.
The article gives no link or citation for it.
The article says “Cloud agents drove 38% of the pull requests we merged in October, up from 0.1% in June”
The source says No source to compare with.
No source givenNumber
Between September and mid-November, the median wait for a first approval fell from over two hours to about five minutes.
The article gives no link or citation for it.
The article says “the median wait for a first approval fell from over two hours to about five minutes”
The source says No source to compare with.
No source givenGeneralization
Changes approved by an agent were reverted no more often than those approved by people.
The article gives no link or citation for it.
The article says “Changes approved by an agent were reverted no more often than those approved by people”
The source says No source to compare with.
No source givenNumber
Across September and October, a person objected to only 15 of the 977 pull requests the AI reviewer approved, about 1.5%.
The article gives no link or citation for it.
The article says “a person objected to only 15 of the 977 pull requests it approved, about 1.5%”
The source says No source to compare with.
No source givenNumber
Of pull requests merged on AI approval alone from October to November, one in a thousand was reverted, versus about one in two hundred for those people approved.
The article gives no link or citation for it.
The article says “only one in a thousand was reverted, compared with about one in two hundred for those people approved”
The source says No source to compare with.
No source givenNumber
In its first four days, the QA loop found 86 confirmed bugs, and agents fixed all of them.
The article gives no link or citation for it.
The article says “In its first four days, the QA loop found 86 confirmed bugs, and agents fixed all of them”
The source says No source to compare with.
No source givenNumber
In its first two weeks the main-watching agents saw about 140 alerts, all but one fixed without calling anyone, with a median of 23 minutes from red to green.
The article gives no link or citation for it.
The article says “In its first two weeks it saw about 140 alerts. All but one were fixed without the coordinator having to call anyone in, with a median of 23 minutes from…”
The source says No source to compare with.
No source givenNumber
In its first two weeks after going live, the coordinator dispatched over 14,000 agent tasks, which led to about 1,900 pull requests.
The article gives no link or citation for it.
The article says “it dispatched over 14,000 agent tasks, from diagnosis and planning to review and fixes, which led to about 1,900 pull requests”
The source says No source to compare with.
Also covered by
No other coverage was clustered with this story.
Elsewhere
Elsewhere: HN 1 pts, not on front page
Recorded for context only, checked 21 Nov 2026, 05:47 UTC. It played no part in the ranking.