TokenGate: Adapting LLMs for pre-fill with learned skimming for ~11x speedup

TokenGate is a drop-in component that adds a learned gate to existing transformer models, letting them select and fully process only the most relevant tokens of a long input context during prefill.