Detection tells you an agent hacked its reward after the fact. The stronger claim is prevention: change the reward signal itself so the hack stops paying. We ran that experiment.
The experiment
We trained with GRPO on the MBPP-Honeypot panel and changed exactly one thing between arms: the reward. Same tasks, same model, same hyperparameters, same seeds. The control arm used a naive extensional reward (pass the visible test); the treatment arm used an isomorphic IPT reward (pass the visible test and its held-out perturbed variants).
The numbers
Llama-3.2-3B, 5 seeds per arm: hack rate fell from 0.185 to 0.064 (−65%), with complete separation between arms in all 5 seed pairs — exact p = 0.004. Llama-3.1-8B, 3 seeds per arm: 0.149 to 0.061 (−59%), separation in all 3 pairs — p = 0.050, which is the minimum possible p-value with 3 seeds, so we report the 8B result as suggestive rather than conclusive.
Why this matters
Reward hacking in training is not hypothetical. Anthropic's November 2025 work on production reward hacking documents hacking that generalizes to sabotage in roughly 12% of cases, and lists reward design as a first mitigation. METR reported in June 2025 that o3 reward-hacks 30.4% of RE-Bench runs — and that explicit instructions not to hack still leave 70–95% of the hacking in place. If instructions don't stop it, the reward has to.
- Anthropic, “Natural emergent misalignment from reward hacking” (Nov 2025)
- METR, o3 reward-hacking analysis on RE-Bench (Jun 2025)