Test and verification technique effectiveness with GPT-6 Astra
Comments
Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules
Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules
Summary
Dan Luu's post reports a follow-up evaluation of how agents use testing and verification techniques, now with GPT-6 Astra. The Zstd implementation benchmark has saturated, with Astra reaching near-full holdout correctness at about $1 per run on medium effort, so the author moved to harder tasks: the IMAP RFC and…