Last year we built megakernels. This year we deleted them: The road to TKCC
Comments
Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules
Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules
Summary
Together AI's engineering team reports that after building "megakernels" that ran an entire model forward pass in one fused GPU kernel, they found most of the gains could be recovered with simpler tools: CUDA Graphs and PDL on top of default Torch Inductor kernels recovered 98% of megakernel performance, and…