The story’s score and claims

Comments on No. 3 of 142 · Robert Joseph George · edition of Wednesday 18 November 2026

When Rounding Breaks Machine Learning

Summary

Robert Joseph George checked all 154 TorchLean operators (162 with variants) against exact results using FloatLib, a Lean library for modelling floating-point arithmetic, and compared them with PyTorch, JAX, TensorFlow, NumPy, CuPy, ONNX Runtime and MLX across roughly 780 million outputs. He reports rounding bugs in several widely used libraries and in TorchLean's own float32 softmax, plus rewrites that fix them. A formula proved correct over the reals can still fail in floating point. The figures are the author's own and come with no independent source.

Comments

Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules