Edition of Tuesday 17 November 2026

Making CI faster and 95% cheaper

Comments on “Making CI faster and 95% cheaper”

Claim-checked. Scored by all three judges, and its checkable claims were compared with the sources it cites. How far stories are checked

Vincent Koc describes how OpenClaw's CI runtime on Blacksmith runners fell from 158,546 hours in a September peak week to 21,395 in a November week (86.5% less), with estimated compute value down 95.5%. Blacksmith sponsored the capacity, and the workload also shrank, so the figures are not attributable to any single fix. The changes: deleting redundant tests, deferring full validation off main pushes, limiting repeat reviews, and selecting tests from the diff. Pull-request-to-merge time was not measured.

Read the original at vincent.build. The judges saw only its text, with the site, the author and link domains removed; they are shown again after ranking.

Why it ranked

No. 16 of 122 with a score of 0.66. The panel rated it highest on technical depth (0.88) and practical relevance (0.85), lowest on significance (0.57). Placed No. 16, No. 28 by score alone: the placement terms moved other stories past it.

AI judgeJudge 2claude-haiku-5-5
Detailed engineering write-up on CI cost reduction, test pruning and adaptive selection with measured figures, though sponsored by a runner vendor and with some unverified claims.
AI judgeJudge 3gpt-5.6-luna
A detailed, technically substantive CI engineering report with measured workload, cost, testing, scheduling, caching, and adaptive-selection changes. Its headline is qualified by explicit confounding caveats, though the 95% figure is not a controlled causal result.

Each judge wrote its line after answering the rubric, with the site and the author still hidden. Every answer behind this is in the table under Score.

Score

0.66total under rubric v2.3

Judges’ mean answer per question, on a scale from 0 to 1. Questions marked minus count against the score.
  • Significance0.57
  • Reader interest0.80
  • Audience breadth0.61
  • Practical relevance0.85
  • Substantiated0.80
  • Technical depth0.88
  • New information0.81
  • Counts against: Promotional0.18
  • Counts against: Overclaim0.42

Each judge’s answer per question (0–1), their mean and spread, and what the question adds to the score (weight × mean).

QuestionJudge 1Judge 2Judge 3MeanSpreadWeightContribution (weight times mean)
SignificanceHow much does this change what people in tech know or can do?0.640.350.720.570.37+0.24+0.137
Reader interestWould a well-informed senior engineer or tech founder want to read this today, and likely pass it on?0.780.700.930.800.23+0.26+0.209
Audience breadthHow many people who build technology does it concern?0.620.500.700.610.20+0.13+0.079
Practical relevanceHow useful is it to someone who builds software or tech products?0.960.700.900.850.26+0.22+0.188
SubstantiatedHow well is it backed? Consider whether it is the primary source, and whether its main claims come with data, code, a demo, a reproducible method or citations.0.950.600.840.800.35+0.05+0.040
Technical depthHow technically substantive is it?0.990.750.910.880.24+0.05+0.044
New informationDoes it contain information absent from the stories covered in the last 7 days (list provided)?0.840.700.880.810.18+0.05+0.040
PromotionalIs it mainly marketing with little substance (a press release, ad or sponsored post with no real information)? A launch post that carries real news is not promotional.0.070.350.120.180.28−0.19−0.034
OverclaimDo the headline or conclusions claim more than the body supports?0.560.450.240.420.32−0.11−0.046
Sum of contributions+0.657
Claim-check penalty0.000
Total, as published0.657

Spread is the highest minus the lowest judge answer. marks a spread above 0.40: the judges disagree, and the story is marked contested.

The badge for this ranking, for the author's page or anyone's: one line of HTML, and it links back here.

Ranked #16 on Rubric, 17 Nov 2026
<a href="https://rubric.hrih.si/editions/2026-11-17/16/"><img src="https://rubric.hrih.si/badge/2026-11-17/16.svg" alt="Ranked #16 on Rubric, 17 Nov 2026" width="245" height="28"></a>

Claim check

Claim check: 3 matches source · 3 no source given · 2 couldn’t check. Each claim is compared with the source the article itself cites; a flag is published only when both checkers agree.

  1. No source givenNumber

    Blacksmith runner usage fell from 158,546 hours in the September peak week to 21,395 hours in the latest week, 86.5% less runtime.

    The article gives no link or citation for it.

    The article says

    “Our Blacksmith runners logged 158,546 hours in the September peak week and 21,395 hours in the latest week: 86.5% less runtime”

    The source says

    No source to compare with.

  2. No source givenNumber

    The estimated value of the compute at Blacksmith's published prices fell 95.5% between the peak week and the latest week.

    The article gives no link or citation for it.

    The article says

    “The estimated value of that compute fell 95.5%”

    The source says

    No source to compare with.

  3. Couldn’t checkQuote

    Peter Steinberger reported that OpenClaw deleted around 400k lines of its own tests without much change in code coverage.

    The source was unreachable, paywalled or too long, or the two checkers disagreed.

    The article says

    “OpenClaw deleted around 400k LOC of its own tests without much change in code coverage”

    The source says

    No passage could be compared.

    Source cited by the article: x.com/steipete/status/2103147927313199260

  4. Matches sourceBenchmark result

    In one matched job of the same 240 test files, the 16-class runner cost 41.7% less ($0.672 vs $1.152) but took 16.5% longer (20m 51s vs 17m 54s) than the 32-class runner.

    The cited source says the same thing.

    The article says

    “41.7% cheaper. 16.5% slower”

    The source says

    “Identical 240-file sample on 32/16-class: completed provider billing units imply $1.152 → $0.672 at the catalog rates; whole-job wall 1,074s → 1,251s”

    Source cited by the article: github.com/openclaw/openclaw/pull/167021

  5. Couldn’t checkNumber

    Three completed PR runs executed only 15 of 4,018, 16 of 4,480 and 97 of 6,895 planned test entries after adaptive selection.

    The source was unreachable, paywalled or too long, or the two checkers disagreed.

    The article says

    “Full inventory → executed4,018 → 1599.63% fewer entries”

    The source says

    No passage could be compared.

    Source cited by the article: github.com/openclaw/openclaw/actions/runs/42312466737

  6. No source givenNumber

    Estimated daily compute cost fell from about $7,738 on 5 November to $1,507 on 7 November, after adaptive selection rolled out around 6–7 November.

    The article gives no link or citation for it.

    The article says

    “Estimated daily compute cost then fell from about $7,738 on 5 November to $1,507 on 7 November”

    The source says

    No source to compare with.

  7. Matches sourceBenchmark result

    Switching the typechecker from tsgo to tsrs cut a local full-typecheck from 447s to 64s, 85.7% less time.

    The cited source says the same thing.

    The article says

    “tsgo → tsrsSupported Linux / Apple Silicon hosts447s → 64s 85.7% less timeLocal”

    The source says

    “pnpm tsgo:all` passed with no errors on both compilers: 64s with tsrs, 447s with `OPENCLAW_TYPECHECKER=tsgo”

    Source cited by the article: github.com/openclaw/openclaw/pull/175711

  8. Matches sourceNumber

    One change packed 95 planned Node shards into 63, and another rebalancing change cut the slowest observed job from 403 seconds to 300.

    The cited source says the same thing.

    The article says

    “one change packed 95 planned Node shards into 63, keeping the gateway and agent suites isolated”

    The source says

    “reducing the normal Node matrix from 95 planned shards to 63”

    Source cited by the article: github.com/openclaw/openclaw/pull/95649

Also covered by

No other coverage was clustered with this story.

Elsewhere

Elsewhere: HN 3 pts, not on front page

Discussion on Hacker News

Recorded for context only, checked 17 Nov 2026, 05:47 UTC. It played no part in the ranking.

Panel for this edition

Four AI agents ranked this edition: three judges scored every story blind to its site and author, and a fourth compared the top stories’ claims with the sources they cite. Summaries are written by AI.

Model versions are pinned for every edition. How ranking works describes rubric v2.3.

Comments

Readers’ comments on this story have a page of their own, ranked by a public rubric rather than by votes.

Comments on “Making CI faster and 95% cheaper”

More on Rubric

From the last 7 editions, chosen by shared topic, score and date.