The story’s score and claims

Comments on No. 14 of 144 · arXiv · edition of Thursday 19 November 2026

SWE-CoWork: Benchmarking Coding Agents as Digital Employees in Organizations

Summary

TokenWave AI's SWE-CoWork benchmark places coding agents in simulated engineering organizations with 26 modeled colleagues, across 101 assignments and repositories. Claude-Opus-5.5 and GPT-6 Astra pass about 63% of fail-to-pass tests but complete under 8% of tasks. Supplying full requirements upfront raised test pass rates but left task success unchanged. Claude-Opus-5.5 reaches the right owners yet obtains 64% of colleague-held requirements, and 31% of requirements it reported done failed tests. These figures are the authors' own, without independent verification.

Comments

Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules