TPS on Kimi K3 with TPU Megakernels
Comments
Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules
Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules
Summary
Inferact released tpu-megakernels, open-source megakernels for TPU v7 written in Pallas. Its Kimi K3 implementation reports over 700 tokens/s with speculative decoding, versus 452 on GB200, and roughly 1.4 to 2× GB200 decode throughput without it at batch sizes 1 to 8.