LLM prefill, what we got wrong
Comments
Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules
Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules
Summary
Draw Things, a Mac media-generation app maker, published a post on LLM prefill on Apple Silicon. They initially expected diffusion-engine know-how to yield fast prompt processing, but found limits: streaming weights from SSD makes prefill roughly 4–5 times slower on M5 Max-class machines, and their analysis shows full-layer loading beats…