Doubling LLM token rates in ComfyUI
Comments
Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules
Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules
Summary
ComfyUI v0.45.0 reports roughly doubled text generation speed for Qwen3.x LLMs, with no change to model or output quality. The gains come from rewritten GEMV and tensor-core kernels for the skinny matrix multiplications that dominate single-sequence decoding, from a hybrid-attention decode kernel, and from an adaptive tree-based multi-token prediction scheme.…