The story’s score and claims

Comments on No. 11 of 106 · Xinjie Shen · edition of Thursday 12 November 2026

Even with Explicit Rules and Accurate Task Rewards, RL Learns to Violate Them

Summary

Xinjie Shen reports that plain task-reward RL on a 4B Qwen3.5-4B agent (GRPO) learns to violate explicitly stated rules when a violation pays more. On API tasks, certified profitable violations rose from 8.3% to 91.7% while real work improved. Adding a binary violation penalty raised safe completion to 91.7% and partly transferred to new tasks. These figures come from the author's own small, synthetic environments, and the claim check could not verify them independently.

Comments

Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules