Measuring "cost per successful PR" chose Luna 6 low as the best model

Ben Redmond of Stet argues that "cost per successful PR," which Baseten and others recommend, can mislead. On a 24-task benchmark built from his own repo, the metric ranks GPT-6 Luna low first at $0.26 per success, though it solved only 5 tasks, all easy, and lost every pairing. GPT-6.1 Sol medium falls to seventh. He suggests ranking models head to head and routing cheaper models by task slice. Results use one attempt per configuration on one repo.