**The Flip Side of RLHF: On-Policy Feedback for Reward Model Self-Supervised Imp

**The Flip Side of RLHF: On-Policy Feedback for Reward Model Self-Supervised Improvement**

> Building strong reward models (RMs) for language model alignment is bottlenecked by the cost and difficulty of acquiring diverse and reliable preference data from human annotation or judge models. It is dramatically worse as the policy evolves beyond the static RM training. Therefore, we propose SAVE (Self-supervised reward model improvement via Value-Anchored On-policy feedback), a framework that grades on-policy re...

**来源信息**
- **来源**：HuggingFace Daily Papers（社区热门论文）
- **分类**：论文
- **发布时间**：2026-06-01 08:00（北京时间）
- **原文**：[打开原文](https://huggingface.co/papers/2605.30888)