TokiRecur – llama.cpp LLM launcher with KV-cache recycling
Comments
Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules
Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules
Summary
TokiRecur is an open-source (MIT) launcher and daemon that adds exact-prefix KV-cache reuse in front of llama.cpp's llama-server, across CUDA, ROCm, SYCL, Vulkan, Metal, OpenVINO and CPU. Reuse is in-memory and token-verified, so outputs match a no-cache run.