Shaping and Monitoring Models Through Their Internal Representations

Goodfire's paper proposes "mechanistic shaping," which adds a training penalty on a model's reliance on, or expression of, an internal concept direction. On chest X-rays, reliance shaping with agent-selected drain features beat balanced-data training. In DPO on OLMo-3-7B, it kept capability gains while sycophancy stayed near baseline, across 7B to 32B models. Expression shaping reportedly curbed Countdown reward hacking in Qwen3.5-9B, though those figures are unchecked. Probes sometimes broke after shaping, but retrained probes recovered the concept in every experiment.