OMLX: Use a second Mac to speed up local LLM prefill 2.9x
Comments
Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules
Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules
Summary
oMLX 0.7.4.dev1, a development release of the open-source local LLM server for Mac, adds a Prefill Accelerator that lets a faster-GPU Mac handle the first layers of prompt processing while a larger-memory Mac runs the rest, with decode speed unchanged.