Neurons as an Interpretable Basis
Comments
Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules
Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules
Summary
This study argues that individual MLP neurons in Qwen3.6-35B-A3B are interpretable units of computation, not the intractable, polysemantic units often assumed. Using activation sparsification, circuit attribution, and agent-driven experiments, the authors report that ablating 92.2% of weak activations raises language-model loss by only 2.9%.