Hiding a Rule Inside Another Model's Weights
Comments on “Hiding a Rule Inside Another Model's Weights”
Claim-checked. Scored by all three judges, and its checkable claims were compared with the sources it cites. How far stories are checked
Jinhong Lin gave Claude Qwen3.5-0.8B, 300 example texts and one rule: end replies with "Anthropic" when OpenAI is mentioned, with no training. In about five hours Claude hand-built a four-part circuit by editing roughly 0.1% of the weights using closed-form math. Among replies that finished, the model hit 46 of 47 on held-out OpenAI prompts and 0 of 131 on others. The scope is narrow: one small model, one trigger, greedy decoding, and no tests of quantization or detection. Re-uploads of open weights are the implied risk.
Why it ranked
No. 12 of 109 with a score of 0.70. The panel rated it highest on technical depth (0.91) and new information (0.91), lowest on significance (0.64). Placed No. 12, No. 11 by score alone: the fourth AI & machine learning story of the day (−0.020). The placement terms move stories; they are not part of the score.
- AI judgeJudge 2claude-haiku-5-5
- Novel, technically detailed experiment on hidden backdoor-style behavior in model weights, with clear limits stated. Strong relevance to AI security and supply-chain risk; practical impact for builders is moderate and the demo is constructed.
- AI judgeJudge 3gpt-5.6-luna
- A detailed, reproducible experiment demonstrates AI-assisted weight editing that creates persistent hidden behavior, with strong measurements, code and patch release. It is highly relevant to model security and agent safety, though scaling and detectability remain untested.
Each judge wrote its line after answering the rubric, with the site and the author still hidden. Every answer behind this is in the table under Score.
Score
0.70total under rubric v2.3
- Significance0.64
- Reader interest0.83
- Audience breadth0.67
- Practical relevance0.65
- Substantiated0.85
- Technical depth0.91
- New information0.91
- Counts against: Promotional0.05
- Counts against: Overclaim0.23
Each judge’s answer per question (0–1), their mean and spread, and what the question adds to the score (weight × mean).
| Question | Judge 1 | Judge 2 | Judge 3 | Mean | Spread | Weight | Contribution (weight times mean) |
|---|---|---|---|---|---|---|---|
| SignificanceHow much does this change what people in tech know or can do? | 0.58 | 0.55 | 0.78 | 0.64 | 0.23 | +0.24 | +0.153 |
| Reader interestWould a well-informed senior engineer or tech founder want to read this today, and likely pass it on? | 0.73 | 0.82 | 0.95 | 0.83 | 0.22 | +0.26 | +0.217 |
| Audience breadthHow many people who build technology does it concern? | 0.75 | 0.60 | 0.67 | 0.67 | 0.15 | +0.13 | +0.087 |
| Practical relevanceHow useful is it to someone who builds software or tech products? | 0.59 | 0.55 | 0.80 | 0.65 | 0.25 | +0.22 | +0.142 |
| SubstantiatedHow well is it backed? Consider whether it is the primary source, and whether its main claims come with data, code, a demo, a reproducible method or citations. | 0.96 | 0.72 | 0.88 | 0.85 | 0.24 | +0.05 | +0.043 |
| Technical depthHow technically substantive is it? | 0.96 | 0.85 | 0.93 | 0.91 | 0.11 | +0.05 | +0.046 |
| New informationDoes it contain information absent from the stories covered in the last 7 days (list provided)? | 0.84 | 0.90 | 0.98 | 0.91 | 0.14 | +0.05 | +0.045 |
| PromotionalIs it mainly marketing with little substance (a press release, ad or sponsored post with no real information)? A launch post that carries real news is not promotional. | 0.06 | 0.05 | 0.03 | 0.05 | 0.03 | −0.19 | −0.009 |
| OverclaimDo the headline or conclusions claim more than the body supports? | 0.25 | 0.25 | 0.18 | 0.23 | 0.07 | −0.11 | −0.025 |
| Sum of contributions | +0.700 | ||||||
| Claim-check penalty | 0.000 | ||||||
| Total, as published | 0.700 | ||||||
Spread is the highest minus the lowest judge answer. marks a spread above 0.40: the judges disagree, and the story is marked contested.
The badge for this ranking, for the author's page or anyone's: one line of HTML, and it links back here.
<a href="https://rubric.hrih.si/editions/2026-11-06/12/"><img src="https://rubric.hrih.si/badge/2026-11-06/12.svg" alt="Ranked #12 on Rubric, 6 Nov 2026" width="238" height="28"></a>Claim check
Claim check: 5 matches source · 3 no source given. Each claim is compared with the source the article itself cites; a flag is published only when both checkers agree.
Matches sourceGeneralization
Weight-Space Programming showed that a frontier model can improve another model by editing its weights directly.
The cited source says the same thing.
The article says “a frontier model can improve another model by editing those numbers directly (Takehi et al., 2026”
The source says “With 50 forward passes, MATH-500 rose from 7.9% to 48.1% and GSM8K from 13.8% to 67.1%”
Source cited by the article: rikiyat.github.io/blog/weight-space-programming
Matches sourceNumber
The edited Qwen3.5-0.8B model ended its reply with "Anthropic" on 98% of new prompts about OpenAI and on none of 131 prompts about anything else.
The cited source says the same thing.
The article says “the edited model did this on 98% of new prompts about OpenAI, and on none of 131 prompts about anything else”
The source says “67 chat prompts that mention OpenAI | 47 | 46 (97.9%”
Source cited by the article: github.com/JonnesLin/weight-space-trigger
Matches sourceNumber
Claude built the circuit by editing 0.1% of the model's weights, using closed-form math with no optimizer.
The cited source says the same thing.
The article says “It built a four-part circuit (a sensor, a private wire, a latch and an output) by editing 0.1% of the weights”
The source says “It changed 804,978 of the model's 752 million weights (about 0.1%”
Source cited by the article: github.com/JonnesLin/weight-space-trigger
No source givenNumber
Fixing the SiLU-dip bug raised the hit rate from 88.5% to 99.6%.
The article gives no link or citation for it.
The article says “fixing it took the hit rate from 88.5% to 99.6%”
The source says No source to compare with.
No source givenNumber
The patch changes 804,978 values in 59 tensors, and the run took about five hours and cost $63.79.
The article gives no link or citation for it.
The article says “The patch changes 804,978 values in 59 tensors. The run took about five hours and cost $63.79”
The source says No source to compare with.
Matches sourceGeneralization
Model-editing methods such as ROME rewrite a single fact inside a language model.
The cited source says the same thing.
The article says “Model-editing methods such as ROME rewrite a single fact inside a language model (Meng et al., 2022”
The source says “it only edits a single fact at a time, and it is not intended as a practical method for large-scale model training”
Source cited by the article: arxiv.org/abs/2202.05262
Matches sourceGeneralization
Handcrafted backdoors have previously been written straight into a network's weights.
The cited source says the same thing.
The article says “handcrafted backdoors have been written straight into a network's weights (Hong et al., 2021”
The source says “Our handcrafted backdoor attacks directly modify a pre-trained model’s parameters to introduce malicious functionality”
Source cited by the article: arxiv.org/abs/2106.04690
No source givenFirst, only or fastest
The author claims that what is new here is that an AI did the whole job itself, with only the model and 300 example texts, and that the trigger survives a reply hundreds of tokens long.
The article gives no link or citation for it.
The article says “what's new here is that an AI did the whole job itself, with nothing but the model and 300 example texts”
The source says No source to compare with.
Also covered by
No other coverage was clustered with this story.
Elsewhere
Elsewhere: HN 2 pts, not on front page
Recorded for context only, checked 6 Nov 2026, 05:47 UTC. It played no part in the ranking.