Training Foundation Models with a Single Algorithm

Researchers at Carnegie Mellon and Johns Hopkins propose "unified training," which treats pretraining, midtraining, SFT and RL as one likelihood-fitting algorithm over a changing data mixture. The reinforcement learning (RL) targets build on MaxRL, and a single trainer mixes static data with RL generations. In a controlled setup (a 124M-parameter model on reasoning puzzles) the best unified configurations beat staged training on accuracy, and they retained more general capability. Results come from one small model and one task family, and staged training was compute-optimal at intermediate budgets.