SlopCodeBench: Benchmarking How Coding Agents Degrade over Long-Horizon Tasks
Comments
Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules
Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules
Summary
SlopCodeBench, from University of Wisconsin–Madison researchers and collaborators, has coding agents extend their own code across 36 problems and 196 checkpoints, with hidden tests and only external behavior specified. Across 15 models, none solves a problem end-to-end, and the best passes 14.8% of checkpoints. By the authors' metrics, erosion rises in 77% of trajectories and verbosity in 75.5%, and agent code is 2.3x more verbose and 2.0x more eroded than 473 Python repositories. Quality prompts lower starting levels, not degradation. These figures are the authors' own and unverified.