The story’s score and claims

Comments on No. 6 of 140 · dangxingyu.github.io · edition of Friday 20 November 2026

A Megakernel per Model

Summary

Xingyu Dang reports that Claude coding agents wrote a single-GPU-program "megakernel" for each of eight open models, from 8B to 594B parameters. At batch size 1, with the reference data flow, the engines run 1.13–1.77× faster than the fastest of vLLM, SGLang or Cohere's own megakernel; allowing lossless storage changes raises this to 2.05×. On a production trace with Inkling-2-Small on 4×B200, 99–100% of requests met their latency targets versus 82–85% for SGLang. Gains shrink at batch size 8, and the post's own figures cover specific models and hardware.

Comments

Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules