Sturnus, a latency-based LLM load balancer
Comments
Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules
Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules
Summary
Sturnus is an open-source, MIT-licensed LLM load balancer distributed as a single static binary that runs as a sidecar beside an application and exposes an OpenAI-compatible API.