I built a tool that finds the fastest way to serve an LLM on your GPU
Comments
Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules
Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules
Summary
rigtune is an open-source tool that benchmarks serving engines (vLLM, SGLang, llama.cpp), weight precisions, KV-cache types and speculative decoding on a user's own GPU and prompts, then serves the fastest configuration via an OpenAI-compatible API.