The story’s score and claims

Comments on No. 29 of 123 · nathan.rs · edition of Saturday 14 November 2026

Eliminating Nondeterminism in LLM Inference with Integer-Only Arithmetic

Summary

DetLLM runs Qwen3-0.6B entirely in exact integer arithmetic, from input ids to int32 logits, so every matmul, softmax, normalization, and rotary embedding is free of floating-point rounding. Floating-point addition is non-associative, so GPU kernels that reorder sums across batch sizes and devices produce different outputs.

Comments

Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules