DeepSeek V3 in ~200 lines of pure Jax
Comments
Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules
Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules
Summary
A solo developer reimplemented the DeepSeek-V3 architecture in about 190 lines of pure JAX, with no Flax, Equinox or PyTorch, and pretrained a 36M-parameter model from scratch on public-domain detective fiction on a MacBook Air.