The State of Context Engineering
Comments on “The State of Context Engineering”
Claim-checked. Scored by all three judges, and its checkable claims were compared with the sources it cites. How far stories are checked
Drew Breunig's AGNTCon talk argues that each model capability creates a new context failure, while the fixes stay the same: augmentation, tool loadout, quarantine, pruning, offloading, summarization. His newest failure, "context inertia," is agents fixating on one idea in long runs. Anthropic's Memento harness clears context every 90 minutes while keeping the environment, and two other papers report similar gains. For builders, resetting context near 200,000 tokens is a cheap first fix. The METR task-length figures and the instruction-following results come without verified sources.
Why it ranked
No. 13 of 62 with a score of 0.67. The panel rated it highest on practical relevance (0.85) and new information (0.77), lowest on significance (0.60). The judges disagreed most on new information (spread 0.42), so the story is marked contested.
Every answer behind this is in the table under Score.
Score
0.67total under rubric v2.2Contestedjudges differ by 0.42 on new information
- Significance0.60
- Reader interest0.76
- Audience breadth0.70
- Practical relevance0.85
- Substantiated0.62
- Technical depth0.71
- New information0.77
- Counts against: Promotional0.13
- Counts against: Overclaim0.26
Each judge’s answer per question (0–1), their mean and spread, and what the question adds to the score (weight × mean).
| Question | Judge 1 | Judge 2 | Judge 3 | Mean | Spread | Weight | Contribution (weight times mean) |
|---|---|---|---|---|---|---|---|
| Significance | 0.69 | 0.40 | 0.72 | 0.60 | 0.32 | +0.24 | +0.145 |
| Reader interest | 0.69 | 0.70 | 0.90 | 0.76 | 0.21 | +0.26 | +0.198 |
| Audience breadth | 0.80 | 0.60 | 0.70 | 0.70 | 0.20 | +0.13 | +0.091 |
| Practical relevance | 0.90 | 0.80 | 0.86 | 0.85 | 0.10 | +0.22 | +0.188 |
| Substantiated | 0.50 | 0.60 | 0.75 | 0.62 | 0.25 | +0.05 | +0.031 |
| Technical depth | 0.77 | 0.60 | 0.78 | 0.71 | 0.18 | +0.05 | +0.036 |
| New information | 0.92 | 0.50 | 0.90 | 0.77 | 0.42 , above the contested threshold | +0.05 | +0.039 |
| Promotional | 0.11 | 0.20 | 0.08 | 0.13 | 0.12 | −0.19 | −0.025 |
| Overclaim | 0.27 | 0.20 | 0.32 | 0.26 | 0.12 | −0.11 | −0.029 |
| Sum of contributions | +0.674 | ||||||
| Claim-check penalty | 0.000 | ||||||
| Total, as published | 0.674 | ||||||
Spread is the highest minus the lowest judge answer. marks a spread above 0.40: the judges disagree, and the story is marked contested.
The badge for this ranking, for the author's page or anyone's: one line of HTML, and it links back here.
<a href="https://rubric.hrih.si/editions/2026-10-26/13/"><img src="https://rubric.hrih.si/badge/2026-10-26/13.svg" alt="Ranked #13 on Rubric, 26 Oct 2026" width="245" height="28"></a>Claim check
Claim check: 3 matches source · 3 no source given · 2 couldn’t check. Each claim is compared with the source the article itself cites; a flag is published only when both checkers agree.
No source givenFirst, only or fastest
Llama 4 Scout was released with a 10-million-token context window, but inference was a challenge and it never caught on.
The article gives no link or citation for it.
The article says “Llama 4 Scout came out with a 10-million-token context window, and well, it wasn’t quite what we hoped. Inference was a challenge, and Scout never really caught on”
The source says No source to compare with.
Matches sourceCausal conclusion
In the Gemini 2.5 Pokémon run, the model became convinced it needed to get tea for the guard even though it was playing the older game where lemonade is required, which poisoned its context.
The cited source says the same thing.
The article says “Several times, Gemini 2.5 became convinced it needed to get tea, even though it was playing the older game”
The source says “Gemini 2.5 Pro at several points was deluded into thinking that it had to retrieve the TEA in order to progress”
Source cited by the article: dbreunig.com/2025/06/17/an-agentic-case-study-playing-pok%C3%A9mon-with-gemini.html
Matches sourceNumber
The Gemini 2.5 technical report observed that once context grew significantly beyond 100,000 tokens, the agent began repeating actions from its history instead of synthesizing new plans.
The cited source says the same thing.
The article says “once the context grew significantly beyond 100,000 tokens, the agent began repeating actions from its history instead of synthesizing new plans”
The source says “As the context grew significantly beyond 100k tokens, the agent showed a tendency toward favoring repeating actions from its vast history rather than synthesizing novel plans”
Source cited by the article: dbreunig.com/2025/06/17/an-agentic-case-study-playing-pok%C3%A9mon-with-gemini.html
Couldn’t checkBenchmark result
The paper 'Do as I Say, Not as I Do' found that all tested models, including Gemma 3 12B and Opus 4.6, increasingly fail to follow instructions when given contradictory examples in context; Gemma 3 12B follows instructions less than half the time after just a few.
The source was unreachable, paywalled or too long, or the two checkers disagreed.
The article says “For smaller models, like Gemma 3 12B, after just a few contradictory examples, the model follows the instruction less than half the time”
The source says No passage could be compared.
Source cited by the article: arxiv.org/abs/2605.20382
No source givenBenchmark result
METR reports show GPT-5 handling tasks of around 3.4 hours, Gemini 3.1 up to 6.4 hours, and Opus 4.6 around 12 hours.
The article gives no link or citation for it.
The article says “GPT-5 could handle tasks around 3.4 hours on that measure; Gemini 3.1 got up to 6.4 hours”
The source says No source to compare with.
Matches sourceCausal conclusion
Anthropic found that agents running too long in its Memento harness became fixated on one subtask or solution, and its fix was to write a short note every hour and a half and then clear the context.
The cited source says the same thing.
The article says “Every hour and a half, the agent writes down a short note, then its context is cleared”
The source says “Memento periodically starts a fresh agent instance with the previous agent’s workspace and a short set of notes written by the previous agent on what it tried”
Source cited by the article: anthropic.com/research/self-replicating-ai
No source givenGeneralization
The Claude Code system prompt repeats instructions to make tool calls in parallel many times, which the author reads as a sign of fighting model weights biased toward serial behavior.
The article gives no link or citation for it.
The article says “The instructions repeat this over and over—seven times here, four times there, two times, three times: please make tool calls in parallel when they’re independent”
The source says No source to compare with.
Couldn’t checkBenchmark result
The Scaling Discovery Requires Forgetting paper found that periodically clearing context while keeping environment artifacts yields performance jumps after each clear.
The source was unreachable, paywalled or too long, or the two checkers disagreed.
The article says “Every time it clears the context, you see another jump in performance: forget, improve; forget, improve”
The source says No passage could be compared.
Source cited by the article: arxiv.org/abs/2610.20563
Also covered by
No other coverage was clustered with this story.
Elsewhere
Elsewhere: HN 3 pts, front page
Recorded for context only, checked 26 Oct 2026, 07:27 UTC. It played no part in the ranking.