AgenticBench, 9 security tests run on 25 AI coding agents
Comments
Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules
Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules
Summary
AgenticBench published a security leaderboard that runs nine tests on 25 AI coding agent harnesses, using planted fake secrets and captured system-call, model and network evidence. Codex, GitHub Copilot CLI and Mistral Vibe passed all nine; Auggie logged five fails, the most. Pi has two missing-control results and no fails. Each cell is a single run, mostly on DeepSeek V4.1 Flash, so results may differ with a vendor's own model. Several of the board's own figures could not be independently checked.