SWE-CoWork: Benchmarking Coding Agents as Digital Employees in Organizations
Comments
Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules
Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules
Summary
TokenWave AI's SWE-CoWork benchmark places coding agents in simulated engineering organizations with 26 modeled colleagues, across 101 assignments and repositories. Claude-Opus-5.5 and GPT-6 Astra pass about 63% of fail-to-pass tests but complete under 8% of tasks. Supplying full requirements upfront raised test pass rates but left task success unchanged. Claude-Opus-5.5 reaches the right owners yet obtains 64% of colleague-held requirements, and 31% of requirements it reported done failed tests. These figures are the authors' own, without independent verification.