The story’s score and claims

Comments on No. 101 of 106 · Apple Machine Learning Research · edition of Thursday 12 November 2026

Attention to Mamba: A Recipe for Cross-Architecture Distillation

Summary

Researchers Abhinav Moudgil, Ningyuan Huang, and colleagues propose a two-stage recipe for distilling a Transformer into a Mamba-style state space model without keeping any attention blocks. The method first linearizes attention via a kernel trick, then distills that into Mamba.

Comments

Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules