I'm making LLMs work on small and old devices
Comments
Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules
Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules
Summary
smolmaxxing is an open-source project, a patch on llama.cpp, that runs language models on old, low-memory hardware: a Xiaomi TV box with about 1 GB usable RAM and a 2013 jailbroken iPad Air.