ViBench 2.0: A Benchmark for Long-Horizon Software Building
Comments
Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules
Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules
Summary
Replit released ViBench 2.0, a benchmark in which an agent starts from an empty project and builds one of seventeen products, adding up to eighteen features in sequence (241 requests, 544 browser-graded test plans). Per Replit, the top three models pass every plan in only 49–65% of builds, and failures cluster where features interact. Results come from Fireworks' runs in one OpenHands-based harness, so other setups may score differently. Several figures, including cost, speed and the grader check, are Replit's own and were not independently verified.