What does LoRA specialized Muon optimizer even mean?
Comments
Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules
Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules
Summary
A solo replication on Qwen3-1.7B with OpenThoughts data tested LoRA-specific optimizers against AdamW and Muon at ranks 16 to 256, using one seed. Anpaure's reverse-engineered Dimuon variant led at every rank, with the margin growing with rank.