Inferssd – Run models larger than RAM or VRAM by streaming MoE experts from SSD
Comments
Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules
Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules
Summary
InferSSD is an Apache-2.0 open-source tool, built on llama.cpp, that runs large mixture-of-experts models such as DeepSeek-V4.1-Flash and Muse-Spark-Open by streaming expert weights from SSD, letting models larger than a machine's RAM or VRAM run on Apple Silicon and NVIDIA Linux systems.