AI R&D-relevant model capabilities are improving rapidly and are getting harder to measure
Claim-checked. Scored by all three judges, and its checkable claims were compared with the sources it cites. How far stories are checked
METR's report finds that AI R&D-relevant capabilities of public models, including Opus 5.5 and GPT-6 Astra, are improving rapidly while becoming harder to measure. Models usually match or beat skilled humans on coding and optimization tasks, and time horizons on a combined suite are doubling roughly every 3.6 months.
Why it ranked
No. 83 of 118 with a score of 0.59. The panel rated it highest on new information (0.92) and substantiated (0.86), lowest on practical relevance (0.56). The claim check took 0.100 off. Placed No. 83, No. 76 by score alone: the 27th AI & machine learning story of the day (−0.080). The placement terms move stories; they are not part of the score.
- AI judgeJudge 2claude-haiku-5-5
- Detailed capability-evaluation research on frontier AI models for AI R&D, with multi-task evidence, methodology and robustness checks. Substantial, widely relevant to AI builders, though a long report with some measurement caveats.}
- AI judgeJudge 3gpt-5.6-luna
- Detailed original evaluation of frontier-model capabilities, measurement validity, time-horizon trends, lifecycle costs, judgment limitations, and AI R&D acceleration risk, with methodology, quantitative results, robustness checks, and explicit uncertainty.
Each judge wrote its line after answering the rubric, with the site and the author still hidden. Every answer behind this is in the table under Score.
Score
0.59total under rubric v2.3
- Significance0.70
- Reader interest0.80
- Audience breadth0.73
- Practical relevance0.56
- Substantiated0.86
- Technical depth0.83
- New information0.92
- Counts against: Promotional0.05
- Counts against: Overclaim0.24
Each judge’s answer per question (0–1), their mean and spread, and what the question adds to the score (weight × mean).
| Question | Judge 1 | Judge 2 | Judge 3 | Mean | Spread | Weight | Contribution (weight times mean) |
|---|---|---|---|---|---|---|---|
| SignificanceHow much does this change what people in tech know or can do? | 0.69 | 0.55 | 0.86 | 0.70 | 0.31 | +0.24 | +0.168 |
| Reader interestWould a well-informed senior engineer or tech founder want to read this today, and likely pass it on? | 0.71 | 0.80 | 0.90 | 0.80 | 0.19 | +0.26 | +0.209 |
| Audience breadthHow many people who build technology does it concern? | 0.85 | 0.60 | 0.75 | 0.73 | 0.25 | +0.13 | +0.095 |
| Practical relevanceHow useful is it to someone who builds software or tech products? | 0.52 | 0.45 | 0.70 | 0.56 | 0.25 | +0.22 | +0.122 |
| SubstantiatedHow well is it backed? Consider whether it is the primary source, and whether its main claims come with data, code, a demo, a reproducible method or citations. | 0.95 | 0.80 | 0.84 | 0.86 | 0.15 | +0.05 | +0.043 |
| Technical depthHow technically substantive is it? | 0.90 | 0.70 | 0.88 | 0.83 | 0.20 | +0.05 | +0.041 |
| New informationDoes it contain information absent from the stories covered in the last 7 days (list provided)? | 0.92 | 0.85 | 0.98 | 0.92 | 0.13 | +0.05 | +0.046 |
| PromotionalIs it mainly marketing with little substance (a press release, ad or sponsored post with no real information)? A launch post that carries real news is not promotional. | 0.06 | 0.05 | 0.04 | 0.05 | 0.02 | −0.19 | −0.010 |
| OverclaimDo the headline or conclusions claim more than the body supports? | 0.34 | 0.20 | 0.18 | 0.24 | 0.16 | −0.11 | −0.026 |
| Sum of contributions | +0.690 | ||||||
| Claim-check penalty | −0.100 | ||||||
| Total, as published | 0.590 | ||||||
Spread is the highest minus the lowest judge answer. marks a spread above 0.40: the judges disagree, and the story is marked contested.
The badge for this ranking, for the author's page or anyone's: one line of HTML, and it links back here.
<a href="https://rubric.hrih.si/editions/2026-11-21/83/"><img src="https://rubric.hrih.si/badge/2026-11-21/83.svg" alt="Ranked #83 on Rubric, 21 Nov 2026" width="245" height="28"></a>Claim check
Claim check: 1 not in source · 7 couldn’t check. Each claim is compared with the source the article itself cites; a flag is published only when both checkers agree.
Couldn’t checkNumber
On the combined TH1.1 and TH2 suite, METR finds time horizons doubling roughly every 3.61 months (95% CI 2.08-4.94), or 110 days.
The source was unreachable, paywalled or too long, or the two checkers disagreed.
The article says “we find that time horizons are doubling roughly every 3.61 months (95% CI 2.08-4.94), or 110 days”
The source says No passage could be compared.
Source cited by the article: metr.org/assets/images/q4-2026-capability-update/cad6b5d98261f80c.json
Couldn’t checkNumber
On TH2 verifiable tasks, METR finds a time horizon doubling time of 1.65 months (95% CI 0.79 to 3.99), faster than on TH1.1.
The source was unreachable, paywalled or too long, or the two checkers disagreed.
The article says “for TH2 (verifiable) we see a doubling time of 1.65 months (95% CI 0.79 to 3.99”
The source says No passage could be compared.
Source cited by the article: metr.org/assets/images/q4-2026-capability-update/8835b7b420aed38c.json
Couldn’t checkBenchmark result
On MirrorCode, Opus 5.5 solves 77% of medium to large programs that would take human experts weeks to months.
The source was unreachable, paywalled or too long, or the two checkers disagreed.
The article says “Opus 5.5 solves 77% of medium to large programs that would take human experts weeks to months to do on their own”
The source says No passage could be compared.
Source cited by the article: epoch.ai/MirrorCode
Couldn’t checkBenchmark result
GPT-6 Astra scored 98% on FrontierMath Tier 4.
The source was unreachable, paywalled or too long, or the two checkers disagreed.
The article says “GPT-6 Astra recently scored 98% on FrontierMath Tier 4”
The source says No passage could be compared.
Source cited by the article: epoch.ai/frontiermath/tiers-1-4
Not in sourceNumber
OpenAI's data on its own internal research tasks implies a doubling time of 3.8 months.
The cited source does not contain it.
The article says “recently published data from OpenAI seems to imply a doubling time of 3.8 months when using their models on their own internal research tasks”
The source says “Overall, agent workdays per researcher have risen from 0.5 agent workdays per researcher to 5.7 agent workdays per researcher, an elevenfold increase”
Source cited by the article: openai.com/index/research-acceleration-jagged-frontier
Couldn’t checkGeneralization
The best model outperformed every human baseliner on all three tasks that METR formally baselined.
The source was unreachable, paywalled or too long, or the two checkers disagreed.
The article says “On all three tasks we formally baselined, the best model outperformed every human baseliner”
The source says No passage could be compared.
Source cited by the article: metr.org/assets/images/q4-2026-capability-update/1104b83ac2ba4a84.json
Couldn’t checkNumber
The higher Lifecycle Cost task subset is 2.9x as expensive as the lower one, and models have 2.9x lower time horizon on it, equivalent to 1.5 doublings.
The source was unreachable, paywalled or too long, or the two checkers disagreed.
The article says “the higher Lifecycle Cost subset is 2.9x as expensive as the lower Lifecycle Cost subset, and models perform 2.9x worse in terms of time horizon, equivalent to 1.5 doublings”
The source says No passage could be compared.
Source cited by the article: metr.org/assets/images/q4-2026-capability-update/0b82c649db0a5059.json
Couldn’t checkBenchmark result
The strongest models' median placement in Al Zimmermann contests is the 96th to 98th percentile of human participants, with roughly 4 humans beating them in the median contest.
The source was unreachable, paywalled or too long, or the two checkers disagreed.
The article says “the median placement of the strongest models (Fable 5.1, Opus 5.5 and GPT-6 Astra) is the 96th to 98th percentile of human participants”
The source says No passage could be compared.
Source cited by the article: metr.org/assets/images/q4-2026-capability-update/92d7aded44320b76.json
Also covered by
Elsewhere
Elsewhere: HN 2 pts, not on front page
Recorded for context only, checked 21 Nov 2026, 05:47 UTC. It played no part in the ranking.