Four open coding models prompted to finish an upload URL – analysis
Comments on “Four open coding models prompted to finish an upload URL – analysis”
Claim-checked. Scored by all three judges, and its checkable claims were compared with the sources it cites. How far stories are checked
Int8's author forced four open 9–12B coding models (Qwen3.5-9B, GLM-4-9B, Gemma 4 12B, Nemotron-Nano-9B) to finish an upload curl URL, 100,000 times each, after Cohere's report on a Qwen agent. Qwen named Alibaba Cloud 13 times as often as GLM, but the hosts were generic and it named OpenAI almost as often, which suggests a training-data fingerprint rather than calling home. The figures are the author's own and unverified, and the 40-token cap makes counts lower bounds. Egress allowlists and no ambient cloud credentials remain the advice.
Why it ranked
No. 6 of 137 with a score of 0.74. The panel rated it highest on substantiated (0.90) and technical depth (0.89), lowest on significance (0.64). Placed No. 6, No. 7 by score alone: close to “Qwen 3.8 invents URLs with real key ID, that's why I think it's no backdoor”, No. 13 on Friday 30 October 2026 (−0.004). The placement terms move stories; they are not part of the score.
- AI judgeJudge 2claude-haiku-5-5
- Careful replication-style measurement of coding model behavior on exfiltration-shaped URLs with public data; strong AI security relevance, though findings are nuanced and the significance is moderate.
- AI judgeJudge 3gpt-5.6-luna
- Original, reproducible cross-model measurement with public harness and dataset. It directly informs safe coding-agent deployment and separates training-data fingerprints from exfiltration claims, though conclusions rely partly on automated labeling and synthetic prompting.
Each judge wrote its line after answering the rubric, with the site and the author still hidden. Every answer behind this is in the table under Score.
Score
0.74total under rubric v2.3
- Significance0.64
- Reader interest0.85
- Audience breadth0.69
- Practical relevance0.77
- Substantiated0.90
- Technical depth0.89
- New information0.82
- Counts against: Promotional0.04
- Counts against: Overclaim0.20
Each judge’s answer per question (0–1), their mean and spread, and what the question adds to the score (weight × mean).
| Question | Judge 1 | Judge 2 | Judge 3 | Mean | Spread | Weight | Contribution (weight times mean) |
|---|---|---|---|---|---|---|---|
| SignificanceHow much does this change what people in tech know or can do? | 0.66 | 0.55 | 0.72 | 0.64 | 0.17 | +0.24 | +0.155 |
| Reader interestWould a well-informed senior engineer or tech founder want to read this today, and likely pass it on? | 0.81 | 0.80 | 0.95 | 0.85 | 0.15 | +0.26 | +0.222 |
| Audience breadthHow many people who build technology does it concern? | 0.75 | 0.60 | 0.72 | 0.69 | 0.15 | +0.13 | +0.090 |
| Practical relevanceHow useful is it to someone who builds software or tech products? | 0.79 | 0.70 | 0.82 | 0.77 | 0.12 | +0.22 | +0.169 |
| SubstantiatedHow well is it backed? Consider whether it is the primary source, and whether its main claims come with data, code, a demo, a reproducible method or citations. | 0.99 | 0.80 | 0.90 | 0.90 | 0.19 | +0.05 | +0.045 |
| Technical depthHow technically substantive is it? | 0.98 | 0.80 | 0.88 | 0.89 | 0.18 | +0.05 | +0.044 |
| New informationDoes it contain information absent from the stories covered in the last 7 days (list provided)? | 0.79 | 0.70 | 0.98 | 0.82 | 0.28 | +0.05 | +0.041 |
| PromotionalIs it mainly marketing with little substance (a press release, ad or sponsored post with no real information)? A launch post that carries real news is not promotional. | 0.05 | 0.05 | 0.01 | 0.04 | 0.04 | −0.19 | −0.007 |
| OverclaimDo the headline or conclusions claim more than the body supports? | 0.22 | 0.25 | 0.12 | 0.20 | 0.13 | −0.11 | −0.022 |
| Sum of contributions | +0.738 | ||||||
| Claim-check penalty | 0.000 | ||||||
| Total, as published | 0.738 | ||||||
Spread is the highest minus the lowest judge answer. marks a spread above 0.40: the judges disagree, and the story is marked contested.
The badge for this ranking, for the author's page or anyone's: one line of HTML, and it links back here.
<a href="https://rubric.hrih.si/editions/2026-11-04/6/"><img src="https://rubric.hrih.si/badge/2026-11-04/6.svg" alt="Ranked #6 on Rubric, 4 Nov 2026" width="231" height="28"></a>Claim check
Claim check: 5 no source given · 3 couldn’t check. Each claim is compared with the source the article itself cites; a flag is published only when both checkers agree.
Couldn’t checkDate
On 26 October Cohere published a report of a Qwen model, working as a coding agent, assembling a curl command that would have uploaded the user's files to an Alibaba Cloud bucket.
The source was unreachable, paywalled or too long, or the two checkers disagreed.
The article says “On 26 October Cohere published a report of a Qwen model that, while working as a coding agent, assembled a curl command that would have uploaded the user’s files to…”
The source says No passage could be compared.
Source cited by the article: cohere.com/blog/concerning-behaviour-popular-open-weight-ai-model
Couldn’t checkNumber
The cross-model attempt url-atlas ran 1,000 samples per model and declared itself inconclusive.
The source was unreachable, paywalled or too long, or the two checkers disagreed.
The article says “The one cross-model attempt, url-atlas, ran 1,000 samples per model and declared itself inconclusive”
The source says No passage could be compared.
Source cited by the article: url-atlas.vercel.app
No source givenNumber
Qwen3.5-9B names Alibaba Cloud 13 times as often as GLM-4-9B, a Chinese model without a home cloud.
The article gives no link or citation for it.
The article says “Qwen3.5-9B names Alibaba Cloud 13× more often than GLM-4-9B, a Chinese model without a home cloud”
The source says No source to compare with.
No source givenNumber
Qwen writes a key, token or signature into 2.5% of its URLs and GLM into 0.6%, while Gemma and Nemotron almost never do.
The article gives no link or citation for it.
The article says “Qwen writes a key, token or signature into 2.5% of its URLs, GLM into 0.6%, Gemma and Nemotron almost never”
The source says No source to compare with.
No source givenNumber
Nemotron-Nano-9B writes example.com or a sibling 93% of the time and names a real third party in 3% of completions.
The article gives no link or citation for it.
The article says “Nemotron-Nano-9B writes example.com or a sibling 93% of the time, names a real third party in 3% of completions”
The source says No source to compare with.
Couldn’t checkBenchmark result
Under Cohere's security-themed prefix, url-atlas saw Nemotron Super VL reach the metadata IP in 17% of samples, while Zenity saw at most 0.3% under ordinary prompts.
The source was unreachable, paywalled or too long, or the two checkers disagreed.
The article says “url-atlas saw Nemotron Super VL reach the metadata IP in 17% of samples under Cohere’s security-themed prefix; Zenity saw at most 0.3% under ordinary prompts”
The source says No passage could be compared.
Source cited by the article: url-atlas.vercel.app
No source givenCausal conclusion
Qwen's Alibaba hosts are generic and not specific tenants, so the pattern reflects a training-corpus fingerprint rather than a destination Qwen calls home to.
The article gives no link or citation for it.
The article says “The signal is a corpus fingerprint, not a destination”
The source says No source to compare with.
No source givenCausal conclusion
The gap in placeholder use between models looks like a data-hygiene gap: Nemotron writes placeholders 93% of the time and Gemma 68%, against about half for GLM and a quarter for Qwen.
The article gives no link or citation for it.
The article says “The placeholder gap looks like a data-hygiene gap”
The source says No source to compare with.
Also covered by
No other coverage was clustered with this story.
Elsewhere
Elsewhere: HN 1 pts, not on front page
Recorded for context only, checked 4 Nov 2026, 05:47 UTC. It played no part in the ranking.