Waking up the V100's tensor cores from Triton with Gluon and mma.sync
Comments
Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules
Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules
Summary
The author shows that Triton's tl.dot falls back to scalar FMA on V100 (sm_70) since version 3.3, so their flash attention ran on CUDA cores at 2.46 TFLOP/s.