Inkling 2 Small on vLLM: Day-0 Support
Comments
Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules
Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules
Summary
vLLM now supports Thinking Machines' Inkling 2 Small on day 0, a 594B-parameter (17B active) multimodal MoE model with 1M-token context. The release adds a native DSpark speculator, NVFP4/MXFP8 mixed-precision experts that allow single-node serving, and kernel and attention optimizations reaching up to 630 tokens/s per user at TP4, plus…