The sleep framework: A new way for LLMs to retain knowledge in continual learning

A new paper proposes a "sleep framework" that alternates LLM deployment with offline phases where the model consolidates short-term context into long-term parameters via expanded mixture-of-experts memory blocks and generates self-practice "dreams" for self-improvement.