BeNice Closing the Inference-Memory Gap of Looped Models with KV-Sharing

Researchers present BeNice!, a post-training method that makes looped language models KV-shareable, meaning one KV-cache can serve all recurrences with near-lossless quality. Pretraining with stochastic depth induces contractive residual dynamics, and BeNice! fixes a first-recurrence basis-shift failure.