How Much of a Harness Does a Strong Agent Need for Autonomous ML Engineering?
Comments
Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules
Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules
Summary
An EPFL and Apple study holds backbone, hardware and time budget fixed and finds that Malena, a single long session of a minimal coding agent, matches or beats four open-source MLE harnesses (MLEvolve, AiScientist, Arbor, ScienceFlow) on MLE-bench and NatureBench at every frontier backbone tested. Execution access is the largest effect; search strategy, delegation and parallelism add no significant gain. The exception is Gemma 4 31B, where MLEvolve leads by mean estimate. Most ablations use one backbone, and several cost and medal-rate figures lack cited sources.