Accelerating Attention for Ultra-Long Context

Prefrontal introduces Latent Sparse Attention (LSA), which organizes keys into a hierarchy of clusters so each query scores only a few clusters instead of every prior token. Distilled into Qwen-3.8-27B, it speeds attention prefill 72.4× at 10M tokens on an H200, or 9.2× end to end. In GLM-5.3-Flash the figures are 22.7× and 2.2×, with RULER scores up to about 8 points lower. The accuracy claims come from the authors' own tests, and the subquadratic cost and the from-scratch results at 128k tokens were not independently checked.