The story’s score and claims

Comments on No. 12 of 46 · GitHub · edition of Monday 9 November 2026

Lossless compression of LLMs to 12-bits

Summary

NF12, an open-source library from dphnAI (Apache-2.0), stores BF16 LLM weights in a lossless 12-bit format, following a 2026 Together AI paper, with CUDA kernels for Ampere through Blackwell. Qwen3-8B shrinks from 15.27 to 11.49 GiB, and Qwen3-30B-A3B from 56.88 to 42.80 GiB. The README's throughput results are mixed: up to 1.34× BF16 on Jetson Thor, but 0.81× on B200 at concurrency 32. These figures are self-reported and could not be independently checked.

Comments

Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules