Sparse kernel delivering 36% higher training throughput
Comments
Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules
Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules
Summary
A new open-source library provides Triton GPU kernels and a PyTorch wrapper for a sparse SwiGLU feed-forward layer, where each token computes only a small fraction of the neurons (6.25% in the author's setup).