Edition of Saturday 21 November 2026

AI R&D-relevant model capabilities are improving rapidly and are getting harder to measure

Comments on “AI R&D-relevant model capabilities are improving rapidly and are getting harder to measure”

Claim-checked. Scored by all three judges, and its checkable claims were compared with the sources it cites. How far stories are checked

METR's report finds that AI R&D-relevant capabilities of public models, including Opus 5.5 and GPT-6 Astra, are improving rapidly while becoming harder to measure. Models usually match or beat skilled humans on coding and optimization tasks, and time horizons on a combined suite are doubling roughly every 3.6 months.

Read the original at metr.org. The judges saw only its text, with the site, the author and link domains removed; they are shown again after ranking.

Why it ranked

No. 83 of 118 with a score of 0.59. The panel rated it highest on new information (0.92) and substantiated (0.86), lowest on practical relevance (0.56). The claim check took 0.100 off. Placed No. 83, No. 76 by score alone: the 27th AI & machine learning story of the day (−0.080). The placement terms move stories; they are not part of the score.

AI judgeJudge 2claude-haiku-5-5
Detailed capability-evaluation research on frontier AI models for AI R&D, with multi-task evidence, methodology and robustness checks. Substantial, widely relevant to AI builders, though a long report with some measurement caveats.}
AI judgeJudge 3gpt-5.6-luna
Detailed original evaluation of frontier-model capabilities, measurement validity, time-horizon trends, lifecycle costs, judgment limitations, and AI R&D acceleration risk, with methodology, quantitative results, robustness checks, and explicit uncertainty.

Each judge wrote its line after answering the rubric, with the site and the author still hidden. Every answer behind this is in the table under Score.

Score

0.59total under rubric v2.3

Judges’ mean answer per question, on a scale from 0 to 1. Questions marked minus count against the score.
  • Significance0.70
  • Reader interest0.80
  • Audience breadth0.73
  • Practical relevance0.56
  • Substantiated0.86
  • Technical depth0.83
  • New information0.92
  • Counts against: Promotional0.05
  • Counts against: Overclaim0.24

Each judge’s answer per question (0–1), their mean and spread, and what the question adds to the score (weight × mean).

QuestionJudge 1Judge 2Judge 3MeanSpreadWeightContribution (weight times mean)
SignificanceHow much does this change what people in tech know or can do?0.690.550.860.700.31+0.24+0.168
Reader interestWould a well-informed senior engineer or tech founder want to read this today, and likely pass it on?0.710.800.900.800.19+0.26+0.209
Audience breadthHow many people who build technology does it concern?0.850.600.750.730.25+0.13+0.095
Practical relevanceHow useful is it to someone who builds software or tech products?0.520.450.700.560.25+0.22+0.122
SubstantiatedHow well is it backed? Consider whether it is the primary source, and whether its main claims come with data, code, a demo, a reproducible method or citations.0.950.800.840.860.15+0.05+0.043
Technical depthHow technically substantive is it?0.900.700.880.830.20+0.05+0.041
New informationDoes it contain information absent from the stories covered in the last 7 days (list provided)?0.920.850.980.920.13+0.05+0.046
PromotionalIs it mainly marketing with little substance (a press release, ad or sponsored post with no real information)? A launch post that carries real news is not promotional.0.060.050.040.050.02−0.19−0.010
OverclaimDo the headline or conclusions claim more than the body supports?0.340.200.180.240.16−0.11−0.026
Sum of contributions+0.690
Claim-check penalty−0.100
Total, as published0.590

Spread is the highest minus the lowest judge answer. marks a spread above 0.40: the judges disagree, and the story is marked contested.

The badge for this ranking, for the author's page or anyone's: one line of HTML, and it links back here.

Ranked #83 on Rubric, 21 Nov 2026
<a href="https://rubric.hrih.si/editions/2026-11-21/83/"><img src="https://rubric.hrih.si/badge/2026-11-21/83.svg" alt="Ranked #83 on Rubric, 21 Nov 2026" width="245" height="28"></a>

Claim check

Claim check: 1 not in source · 7 couldn’t check. Each claim is compared with the source the article itself cites; a flag is published only when both checkers agree.

  1. Couldn’t checkNumber

    On the combined TH1.1 and TH2 suite, METR finds time horizons doubling roughly every 3.61 months (95% CI 2.08-4.94), or 110 days.

    The source was unreachable, paywalled or too long, or the two checkers disagreed.

    The article says

    “we find that time horizons are doubling roughly every 3.61 months (95% CI 2.08-4.94), or 110 days”

    The source says

    No passage could be compared.

    Source cited by the article: metr.org/assets/images/q4-2026-capability-update/cad6b5d98261f80c.json

  2. Couldn’t checkNumber

    On TH2 verifiable tasks, METR finds a time horizon doubling time of 1.65 months (95% CI 0.79 to 3.99), faster than on TH1.1.

    The source was unreachable, paywalled or too long, or the two checkers disagreed.

    The article says

    “for TH2 (verifiable) we see a doubling time of 1.65 months (95% CI 0.79 to 3.99”

    The source says

    No passage could be compared.

    Source cited by the article: metr.org/assets/images/q4-2026-capability-update/8835b7b420aed38c.json

  3. Couldn’t checkBenchmark result

    On MirrorCode, Opus 5.5 solves 77% of medium to large programs that would take human experts weeks to months.

    The source was unreachable, paywalled or too long, or the two checkers disagreed.

    The article says

    “Opus 5.5 solves 77% of medium to large programs that would take human experts weeks to months to do on their own”

    The source says

    No passage could be compared.

    Source cited by the article: epoch.ai/MirrorCode

  4. Couldn’t checkBenchmark result

    GPT-6 Astra scored 98% on FrontierMath Tier 4.

    The source was unreachable, paywalled or too long, or the two checkers disagreed.

    The article says

    “GPT-6 Astra recently scored 98% on FrontierMath Tier 4”

    The source says

    No passage could be compared.

    Source cited by the article: epoch.ai/frontiermath/tiers-1-4

  5. Not in sourceNumber

    OpenAI's data on its own internal research tasks implies a doubling time of 3.8 months.

    The cited source does not contain it.

    The article says

    “recently published data from OpenAI seems to imply a doubling time of 3.8 months when using their models on their own internal research tasks”

    The source says

    “Overall, agent workdays per researcher have risen from 0.5 agent workdays per researcher to 5.7 agent workdays per researcher, an elevenfold increase”

    Source cited by the article: openai.com/index/research-acceleration-jagged-frontier

  6. Couldn’t checkGeneralization

    The best model outperformed every human baseliner on all three tasks that METR formally baselined.

    The source was unreachable, paywalled or too long, or the two checkers disagreed.

    The article says

    “On all three tasks we formally baselined, the best model outperformed every human baseliner”

    The source says

    No passage could be compared.

    Source cited by the article: metr.org/assets/images/q4-2026-capability-update/1104b83ac2ba4a84.json

  7. Couldn’t checkNumber

    The higher Lifecycle Cost task subset is 2.9x as expensive as the lower one, and models have 2.9x lower time horizon on it, equivalent to 1.5 doublings.

    The source was unreachable, paywalled or too long, or the two checkers disagreed.

    The article says

    “the higher Lifecycle Cost subset is 2.9x as expensive as the lower Lifecycle Cost subset, and models perform 2.9x worse in terms of time horizon, equivalent to 1.5 doublings”

    The source says

    No passage could be compared.

    Source cited by the article: metr.org/assets/images/q4-2026-capability-update/0b82c649db0a5059.json

  8. Couldn’t checkBenchmark result

    The strongest models' median placement in Al Zimmermann contests is the 96th to 98th percentile of human participants, with roughly 4 humans beating them in the median contest.

    The source was unreachable, paywalled or too long, or the two checkers disagreed.

    The article says

    “the median placement of the strongest models (Fable 5.1, Opus 5.5 and GPT-6 Astra) is the 96th to 98th percentile of human participants”

    The source says

    No passage could be compared.

    Source cited by the article: metr.org/assets/images/q4-2026-capability-update/92d7aded44320b76.json

Also covered by

Elsewhere

Elsewhere: HN 2 pts, not on front page

Discussion on Hacker News

Recorded for context only, checked 21 Nov 2026, 05:47 UTC. It played no part in the ranking.

Panel for this edition

Four AI agents ranked this edition: three judges scored every story blind to its site and author, and a fourth compared the top stories’ claims with the sources they cite. Summaries are written by AI.

Model versions are pinned for every edition. How ranking works describes rubric v2.3.

Comments

Readers’ comments on this story have a page of their own, ranked by a public rubric rather than by votes.

Comments on “AI R&D-relevant model capabilities are improving rapidly and are getting harder to measure”

More on Rubric

From the last 7 editions, chosen by shared topic, score and date.