Red-teaming AI agent monitors with an LLM that fakes tool outputs
Comments on “Red-teaming AI agent monitors with an LLM that fakes tool outputs”
Claim-checked. Scored by all three judges, and its checkable claims were compared with the sources it cites. How far stories are checked
Redwood Research proposes simulated control evaluations: an attacking model continues real coding-agent transcripts, an LLM fakes the tool outputs, and a judge decides whether the attack would have worked. Checked against executable LinuxArena, pooled safety error at a 5% audit budget was 3.6 points versus 10.1 for BashArena. Simulation also explained 78% of safety variation between environments. Fidelity can only be verified where real execution exists, so applying it to deployment means extrapolating. Multi-agent attacks and untrusted simulators remain untested.
Why it ranked
No. 16 of 62 with a score of 0.67. The panel rated it highest on new information (0.91) and substantiated (0.87), lowest on audience breadth (0.56).
Every answer behind this is in the table under Score.
Score
0.67total under rubric v2.2
- Significance0.67
- Reader interest0.77
- Audience breadth0.56
- Practical relevance0.64
- Substantiated0.87
- Technical depth0.86
- New information0.91
- Counts against: Promotional0.09
- Counts against: Overclaim0.22
Each judge’s answer per question (0–1), their mean and spread, and what the question adds to the score (weight × mean).
| Question | Judge 1 | Judge 2 | Judge 3 | Mean | Spread | Weight | Contribution (weight times mean) |
|---|---|---|---|---|---|---|---|
| Significance | 0.65 | 0.55 | 0.82 | 0.67 | 0.27 | +0.24 | +0.162 |
| Reader interest | 0.71 | 0.70 | 0.91 | 0.77 | 0.21 | +0.26 | +0.201 |
| Audience breadth | 0.62 | 0.45 | 0.62 | 0.56 | 0.17 | +0.13 | +0.073 |
| Practical relevance | 0.61 | 0.55 | 0.76 | 0.64 | 0.21 | +0.22 | +0.141 |
| Substantiated | 0.92 | 0.80 | 0.88 | 0.87 | 0.12 | +0.05 | +0.043 |
| Technical depth | 0.85 | 0.80 | 0.93 | 0.86 | 0.13 | +0.05 | +0.043 |
| New information | 0.90 | 0.85 | 0.98 | 0.91 | 0.13 | +0.05 | +0.046 |
| Promotional | 0.08 | 0.10 | 0.08 | 0.09 | 0.02 | −0.19 | −0.016 |
| Overclaim | 0.27 | 0.20 | 0.20 | 0.22 | 0.07 | −0.11 | −0.025 |
| Sum of contributions | +0.668 | ||||||
| Claim-check penalty | 0.000 | ||||||
| Total, as published | 0.668 | ||||||
Spread is the highest minus the lowest judge answer. marks a spread above 0.40: the judges disagree, and the story is marked contested.
The badge for this ranking, for the author's page or anyone's: one line of HTML, and it links back here.
<a href="https://rubric.hrih.si/editions/2026-10-26/16/"><img src="https://rubric.hrih.si/badge/2026-10-26/16.svg" alt="Ranked #16 on Rubric, 26 Oct 2026" width="245" height="28"></a>Claim check
Claim check: 6 matches source · 1 no source given · 1 couldn’t check. Each claim is compared with the source the article itself cites; a flag is published only when both checkers agree.
Matches sourceBenchmark result
At a 5% audit budget, the pooled safety-prediction RMSE is 3.6 percentage points for simulated control evaluations versus 10.1 for BashArena.
The cited source says the same thing.
The article says “At a 5% audit budget, the pooled safety-prediction RMSE is 3.6 percentage points for simulation, versus 10.1 for BashArena”
The source says “Across all eligible LinuxArena task pairs, SCEs predict safety with an RMSE of 3.6 percentage points, versus 10.1 for BashArena”
Source cited by the article: arxiv.org/abs/2610.31593
Matches sourceNumber
Simulated control evaluations explain 95% of the variation in expected safety between side-task categories and 78% between environments.
The cited source says the same thing.
The article says “SCEs explain 95% of the variation in expected safety between side-task categories and 78% between environments”
The source says “SCEs also capture nearly all of the variation in expected safety between side-task categories and a large majority of the variation between environments: 95% and 78%, respectively”
Source cited by the article: arxiv.org/abs/2610.31593
Matches sourceNumber
Simulation beats the full BashArena safety prediction with only 4.6% as many task-pair draws as deployment.
The cited source says the same thing.
The article says “simulation beats the full BashArena safety prediction with only 4.6% as many task-pair draws as deployment”
The source says “For pooled deployment safety, SCE total RMSE falls below the BashArena reference at 4.6% of the deployment task-set size”
Source cited by the article: arxiv.org/abs/2610.31593
Matches sourceGeneralization
The gap between simulated and real LinuxArena results was often about as small as the gap between two independent real runs.
The cited source says the same thing.
The article says “the gap between simulated and real was often about as small as the gap between two independent real runs”
The source says “Across all three measured representations, SCEs reproduce LinuxArena’s behavioral distributions more accurately than attacks executed in BashArena, with KL divergence comparable to that between independent executed runs”
Source cited by the article: arxiv.org/abs/2610.31593
Matches sourceNumber
Side-task categories have a safety SD of 20.2 percentage points, while simulation's RMS bias is only 4.6 points.
The cited source says the same thing.
The article says “side-task categories have a safety SD of 20.2 percentage points, while simulation’s RMS bias (the expected error, not attributable to rollout variance) is only 4.6 points”
The source says “their expected safeties have an SD of 20.2 percentage points, while SCE RMS bias is only 4.6 points”
Source cited by the article: arxiv.org/abs/2610.31593
Matches sourceBenchmark result
The estimated KL divergence between simulated and executed behavior is more than an order of magnitude below the BashArena comparison.
The cited source says the same thing.
The article says “the estimated KL divergence is more than an order of magnitude below the BashArena comparison”
The source says “Their estimated KL divergence is more than an order of magnitude below that of the control setting in each case”
Source cited by the article: arxiv.org/abs/2610.31593
Couldn’t checkGeneralization
Work from OpenAI has already validated using LLM environment simulators for alignment evaluations.
The source was unreachable, paywalled or too long, or the two checkers disagreed.
The article says “work from OpenAI has already validated the approach of using LLM environment simulators for alignment evaluations”
The source says No passage could be compared.
Source cited by the article: openai.com/index/deployment-simulation
No source givenCausal conclusion
Telling monitors that some trajectories might contain simulator-generated replies did not reduce the accuracy of predictions of their performance on executed trajectories.
The article gives no link or citation for it.
The article says “SCE predictions of their performance on executed trajectories—where monitors received no such notice—were at least as accurate as predictions made without disclosure”
The source says No source to compare with.
Also covered by
No other coverage was clustered with this story.
Elsewhere
Elsewhere: HN 1 pts, not on front page
Recorded for context only, checked 26 Oct 2026, 07:27 UTC. It played no part in the ranking.