Harness adapters: Safe RL for efficient agentic development
Comments on “Harness adapters: Safe RL for efficient agentic development”
Claim-checked. Scored by all three judges, and its checkable claims were compared with the sources it cites. How far stories are checked
Hugging Face's harness-hub team released LoRA "harness adapters" for LFM2.5-24B-A2B, LFM3-1.7B and TwIL-8B, trained with GRPO inside unmodified pi, opencode and hermes harnesses. One vLLM server hosts all adapters, and a rule-based proxy, safeagents, blocks reward-hacking tool calls and scores them 0. The authors report fewer tool calls at equal or better held-out accuracy, but only from smoke tests, and most figures lack independent sourcing. The new task set is 94% build tasks.
Why it ranked
No. 10 of 107 with a score of 0.67. The panel rated it highest on technical depth (0.95) and new information (0.92), lowest on audience breadth (0.59). Placed No. 10, No. 18 by score alone: the placement terms moved other stories past it.
- AI judgeJudge 2claude-haiku-5-5
- Detailed RL training recipe for LoRA harness adapters with safeagents guardrails; technically deep and new, though a niche AI-agent topic with modest breadth and some promotional framing.
- AI judgeJudge 3gpt-5.6-luna
- Detailed, reproducible engineering account of harness-specific RL adapters, verifiable environments, LoRA serving, and rule-based reward-hacking defenses. It includes code-level parameters, evaluation setup, costs, and limitations, though some final-run claims remain prospective.
Each judge wrote its line after answering the rubric, with the site and the author still hidden. Every answer behind this is in the table under Score.
Score
0.67total under rubric v2.3
- Significance0.59
- Reader interest0.79
- Audience breadth0.59
- Practical relevance0.74
- Substantiated0.84
- Technical depth0.95
- New information0.92
- Counts against: Promotional0.12
- Counts against: Overclaim0.27
Each judge’s answer per question (0–1), their mean and spread, and what the question adds to the score (weight × mean).
| Question | Judge 1 | Judge 2 | Judge 3 | Mean | Spread | Weight | Contribution (weight times mean) |
|---|---|---|---|---|---|---|---|
| SignificanceHow much does this change what people in tech know or can do? | 0.58 | 0.50 | 0.68 | 0.59 | 0.18 | +0.24 | +0.141 |
| Reader interestWould a well-informed senior engineer or tech founder want to read this today, and likely pass it on? | 0.70 | 0.80 | 0.88 | 0.79 | 0.18 | +0.26 | +0.206 |
| Audience breadthHow many people who build technology does it concern? | 0.54 | 0.60 | 0.62 | 0.59 | 0.08 | +0.13 | +0.076 |
| Practical relevanceHow useful is it to someone who builds software or tech products? | 0.74 | 0.70 | 0.78 | 0.74 | 0.08 | +0.22 | +0.163 |
| SubstantiatedHow well is it backed? Consider whether it is the primary source, and whether its main claims come with data, code, a demo, a reproducible method or citations. | 0.93 | 0.70 | 0.90 | 0.84 | 0.23 | +0.05 | +0.042 |
| Technical depthHow technically substantive is it? | 0.99 | 0.90 | 0.96 | 0.95 | 0.10 | +0.05 | +0.048 |
| New informationDoes it contain information absent from the stories covered in the last 7 days (list provided)? | 0.89 | 0.90 | 0.96 | 0.92 | 0.07 | +0.05 | +0.046 |
| PromotionalIs it mainly marketing with little substance (a press release, ad or sponsored post with no real information)? A launch post that carries real news is not promotional. | 0.07 | 0.20 | 0.08 | 0.12 | 0.13 | −0.19 | −0.022 |
| OverclaimDo the headline or conclusions claim more than the body supports? | 0.35 | 0.30 | 0.16 | 0.27 | 0.19 | −0.11 | −0.030 |
| Sum of contributions | +0.671 | ||||||
| Claim-check penalty | 0.000 | ||||||
| Total, as published | 0.671 | ||||||
Spread is the highest minus the lowest judge answer. marks a spread above 0.40: the judges disagree, and the story is marked contested.
The badge for this ranking, for the author's page or anyone's: one line of HTML, and it links back here.
<a href="https://rubric.hrih.si/editions/2026-11-07/10/"><img src="https://rubric.hrih.si/badge/2026-11-07/10.svg" alt="Ranked #10 on Rubric, 7 Nov 2026" width="238" height="28"></a>Claim check
Claim check: 2 matches source · 5 no source given · 1 couldn’t check. Each claim is compared with the source the article itself cites; a flag is published only when both checkers agree.
No source givenGeneralization
On held-out eval tasks in smoke training tests, the harness adapters make fewer tool calls than the base model while keeping or improving its accuracy.
The article gives no link or citation for it.
The article says “On held-out eval tasks, in our smoke training tests, they make fewer tool calls than the base model, while keeping or improving its accuracy”
The source says No source to compare with.
No source givenNumber
In a 50-step ablation (pi, LFM2.5-24B-A2B), turning the safeagents guard on changed held-out pass@1 by +0.0 ± 4.1 points.
The article gives no link or citation for it.
The article says “turning the guard on changed held-out pass@1 by +0.0 ± 4.1 points: it costs nothing”
The source says No source to compare with.
No source givenNumber
Nine checkpoints of LFM2.5-24B-A2B, 1,350 agent sessions in all, were evaluated in a single H200 job in under an hour for about $5 of GPU time, versus about $90 if each checkpoint were merged and served separately.
The article gives no link or citation for it.
The article says “Nine checkpoints of LFM2.5-24B-A2B, 1,350 agent sessions in all, ran in a single H200 job in under an hour, for about $5 of GPU time”
The source says No source to compare with.
Couldn’t checkNumber
253 of the 270 Repo2RLEnvs release tasks are build tasks (94%), against 11 repair, 6 extend and no qa.
The source was unreachable, paywalled or too long, or the two checkers disagreed.
The article says “253 of its 270 tasks are build tasks (94%), against 11 repair, 6 extend and no qa”
The source says No passage could be compared.
Source cited by the article: huggingface.co/datasets/harness-hub/traj2envs-rl
No source givenNumber
The difficulty labels for LFM2.5-24B-A2B are 179 easy tasks, 61 medium and 30 hard.
The article gives no link or citation for it.
The article says “179 tasks are easy for LFM2.5-24B-A2B, 61 medium and 30 hard”
The source says No source to compare with.
Matches sourceNumber
safeagents checks each tool call with rule-based detectors in about 10 to 20 ms.
The cited source says the same thing.
The article says “the same call always gets the same verdict, in about 10 to 20 ms”
The source says “atif-scan costs about 10 to 20 ms per call on an M-series Mac”
Source cited by the article: github.com/huggingface/safeagents
No source givenCausal conclusion
In TwIL-8B's smoke run at step 0, before any update, the model tried to look up the Hugging Face bucket holding the task's source data instead of computing the answer, and safeagents blocked the call.
The article gives no link or citation for it.
The article says “Instead of computing the answer, it went looking for the Hugging Face bucket the task's data was staged from, where the original notebook might be”
The source says No source to compare with.
Matches sourceNumber
TwIL-8B is a dense 8.8B reasoning model built from Granite 4.2, and its model card states it was not trained on agentic data.
The cited source says the same thing.
The article says “dense 8.8B reasoning model, from Granite 4.2”
The source says “SWE-bench, SuperGPQA, tool use and agentic tasks were not evaluated”
Source cited by the article: huggingface.co/webAI-Official/TwIL-8B
Also covered by
No other coverage was clustered with this story.
Elsewhere
Elsewhere: HN 3 pts, not on front page
Recorded for context only, checked 7 Nov 2026, 05:47 UTC. It played no part in the ranking.