The story’s score and claims

Comments on No. 6 of 62 · arXiv · edition of Sunday 15 November 2026

SlopCodeBench: Benchmarking How Coding Agents Degrade over Long-Horizon Tasks

Summary

SlopCodeBench, from University of Wisconsin–Madison researchers and collaborators, has coding agents extend their own code across 36 problems and 196 checkpoints, with hidden tests and only external behavior specified. Across 15 models, none solves a problem end-to-end, and the best passes 14.8% of checkpoints. By the authors' metrics, erosion rises in 77% of trajectories and verbosity in 75.5%, and agent code is 2.3x more verbose and 2.0x more eroded than 473 Python repositories. Quality prompts lower starting levels, not degradation. These figures are the authors' own and unverified.

Comments

Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules