Eliminating Nondeterminism in LLM Inference with Integer-Only Arithmetic
Comments
Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules
Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules
Summary
DetLLM runs Qwen3-0.6B entirely in exact integer arithmetic, from input ids to int32 logits, so every matmul, softmax, normalization, and rotary embedding is free of floating-point rounding. Floating-point addition is non-associative, so GPU kernels that reorder sums across batch sizes and devices produce different outputs.