The Surprising Effectiveness of Deleting Weights in LLM Post-Training
Comments
Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules
Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules
Summary
Researchers at New York University and the University of Calgary (Yang, Lasby, Pote, Ioannou, Ren) propose Deletion Tuning, which post-trains a pretrained LLM by learning only a binary mask that zeroes fewer than 0.05% of gate-projection weights.