The Curse of Recursion: Training on Generated Data Makes Models Forget (2024)
Comments
Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules
Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules
Summary
Researchers at Cambridge, Oxford, Imperial, Toronto and Edinburgh show that models trained recursively on their predecessors' output lose the tails of the original distribution and drift toward a narrow point estimate. They demonstrate this in Gaussian mixtures, variational autoencoders and fine-tuned language models, and argue from finite sampling error that it is unavoidable. For teams that train on scraped web data, human-generated data and provenance become more valuable. The language-model results come from fine-tuning, where only early signs appear, and several quoted figures lacked sources.