The story’s score and claims

Comments on No. 42 of 107 · zaage.IT · edition of Saturday 7 November 2026

Swapping LLMs in Under Two Seconds: A Sleep Mode for Llama.cpp

Summary

A developer's llama-sleep wrapper for llama-swap parks idle llama.cpp models' GPU memory in host RAM instead of restarting them, cutting a 10.3-second model swap on a single RTX 3060 to 2.1 seconds while keeping cached prompts.

Comments

Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules