I let LLMs play a 1988 detective gamebook 646 times
Comments
Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules
Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules
Summary
A developer ran 95 runs and 646 games of LLMs playing a 1988 detective gamebook using the GitHub Copilot SDK. Every first attempt lost, but capable models learned from retries, with more attempts outperforming a bigger model.