Red-Teaming OpenAI's Internal Agent Monitoring Systems
Claim-checked. Scored by all three judges, and its checkable claims were compared with the sources it cites. How far stories are checked
METR and Redwood Research staff spent three weeks in July and August with deep internal access to OpenAI, testing its asynchronous misalignment monitor as it stood on July 21. They estimate 75–94% of agentic Responses API traffic was eligible for monitoring, and roughly 10% of eligible Codex traffic went unmonitored through compaction gaps. They also found client-controllable headers that disable monitoring and serialization mismatches that could make the monitor fail. OpenAI says the issues are patched; the authors have not verified this, and the public write-up is a redacted summary.
Score
0.74total under rubric v2.0
- Significance0.71
- Reader interest0.87
- Audience breadth0.55
- Practical relevance0.74
- Substantiated0.80
- Technical depth0.76
- New information0.96
- Counts against: Promotional0.05
- Counts against: Overclaim0.19
Each judge’s answer per question (0–1), their mean and spread, and what the question adds to the score (weight × mean).
| Question | Judge 1 | Judge 2 | Judge 3 | Mean | Spread | Weight | Contribution (weight times mean) |
|---|---|---|---|---|---|---|---|
| Significance | 0.68 | 0.60 | 0.86 | 0.71 | 0.26 | +0.25 | +0.178 |
| Reader interest | 0.73 | 0.90 | 0.97 | 0.87 | 0.24 | +0.20 | +0.173 |
| Audience breadth | 0.34 | 0.60 | 0.72 | 0.55 | 0.38 | +0.10 | +0.055 |
| Practical relevance | 0.70 | 0.70 | 0.83 | 0.74 | 0.13 | +0.15 | +0.111 |
| Substantiated | 0.78 | 0.75 | 0.87 | 0.80 | 0.12 | +0.10 | +0.080 |
| Technical depth | 0.75 | 0.70 | 0.84 | 0.76 | 0.14 | +0.10 | +0.076 |
| New information | 0.93 | 0.95 | 0.99 | 0.96 | 0.06 | +0.10 | +0.096 |
| Promotional | 0.07 | 0.05 | 0.02 | 0.05 | 0.05 | −0.15 | −0.007 |
| Overclaim | 0.29 | 0.15 | 0.12 | 0.19 | 0.17 | −0.15 | −0.028 |
| Sum of contributions | +0.735 | ||||||
| Claim-check penalty | 0.000 | ||||||
| Total, as published | 0.735 | ||||||
Spread is the highest minus the lowest judge answer. marks a spread above 0.40: the judges disagree, and the story is marked contested.
Claim check
Claim check: 7 no source given · 1 couldn’t check. Each claim is compared with the source the article itself cites; a flag is published only when both checkers agree.
No source givenDate
A METR employee and a Redwood Research employee conducted a three-week pilot high-access exercise, from July 21st to August 11th, to stress-test OpenAI's internal misalignment monitoring system.
The article gives no link or citation for it.
The article says “For three weeks, from July 21st to August 11th, a METR employee and a Redwood Research employee conducted a pilot high-access exercise”
The source says No source to compare with.
No source givenNumber
The exercise produced a 61-page internal report that was shared with OpenAI researchers and security staff.
The article gives no link or citation for it.
The article says “The output of this exercise was a 61-page internal report that was shared with OpenAI researchers and security staff”
The source says No source to compare with.
No source givenNumber
METR and Redwood estimated that roughly 75–94% of agentic internal Responses API traffic from models at least as strong as GPT-5.5 was eligible for monitoring at the time of the exercise.
The article gives no link or citation for it.
The article says “roughly 75–94% of agentic internal Responses API (which powers Codex, among other things)4 traffic from models at least as strong as GPT-5.5 was eligible for monitoring”
The source says No source to compare with.
No source givenNumber
Of the Responses API traffic used for internal Codex agents, 99.2% was eligible for monitoring.
The article gives no link or citation for it.
The article says “out of the subset of Responses API traffic used for internal Codex agents, 99.2% was eligible for monitoring”
The source says No source to compare with.
No source givenCausal conclusion
Not monitoring compaction events likely led to on the order of 10% of eligible internal Codex traffic not being monitored between May and July 2026.
The article gives no link or citation for it.
The article says “not monitoring compaction events likely led to on the order of 10% of eligible internal Codex traffic not being monitored between May and July 2026”
The source says No source to compare with.
No source givenNumber
Information given to the researchers suggested the monitor errored on less than 0.2% of requests, with zero load-sheds in one 24-hour period during the exercise.
The article gives no link or citation for it.
The article says “the monitor errored on less than 0.2% of requests, and that in one 24-hour period during the exercise there were zero load-sheds”
The source says No source to compare with.
No source givenNumber
The researchers found two independent client-controllable headers that disable the monitoring system for new agents.
The article gives no link or citation for it.
The article says “We found two independent client-controllable headers that disable the monitoring system for new agents”
The source says No source to compare with.
Couldn’t checkQuote
OpenAI has stated that all vulnerabilities identified that affect its production safeguards stack have been patched.
The source was unreachable, paywalled or too long, or the two checkers disagreed.
The article says “OpenAI has stated that all of the vulnerabilities we identified that impact their production safeguards stack have been patched”
The source says No passage could be compared.
Source cited by the article: openai.com/index/hugging-face-incident-and-the-road-ahead
Also covered by
Elsewhere
Elsewhere: HN 1 pts, not on front page
Recorded for context only, checked 24 Oct 2026, 08:53 UTC. It played no part in the ranking.