The story’s score and claims

Comments on No. 111 of 142 · sky-posttrain.github.io · edition of Wednesday 18 November 2026

Beyond Score Centering: What Else Can We Optimize in Async RL?

Summary

Score centering (SC) addresses instability in asynchronous reinforcement learning for LLMs, where the trainer learns from trajectories sampled by a policy several steps old. The post shows that the policy-gradient norm grows roughly linearly with trajectory age, because an off-policy drift term appears.

Comments

Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules