**How Much Training-Inference Mismatch Can On-Policy Distillation Tolerate?** >

**How Much Training-Inference Mismatch Can On-Policy Distillation Tolerate?**

> On-policy distillation (OPD) trains a student language model on its own responses, guided by a fixed teacher. Generating these responses is expensive, so systems use quantized inference and asynchronous rollouts to reduce the cost. Both, however, introduce training-inference mismatch between the rollout engine and the model being trained. Such mismatch can destabilize reinforcement learning (RL), but its effects on O...

**来源信息**
- **来源**：HuggingFace Daily Papers（社区热门论文）
- **分类**：论文
- **发布时间**：2026-11-10 08:00（北京时间）
- **原文**：[打开原文](https://huggingface.co/papers/2611.11450)