Together AI Compiler for Inference
Comments
Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules
Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules
Summary
Together AI has introduced Together Compiler (TKCC), an agent-driven system that continually re-optimizes LLM inference as traffic, models, and hardware change. It uses persistent engines that hot-reload candidate kernels instead of restarting, reporting up to 2.6x higher TPS on frontier open models and 1.7x over a Llama 1B megakernel baseline.