**Metric-Dependent Annotation Saturation for Learning from Label Distributions**

**Metric-Dependent Annotation Saturation for Learning from Label Distributions**

> When annotators disagree on a label, the disagreement itself carries signal—and the number of annotators needed to capture it depends on the evaluation metric. We fine-tune NLI models on label distributions subsampled from ChaosNLI, a dataset providing 100 independent annotator judgments per item, and identify metric-dependent saturation. In our 3-class NLI setting, entropy correlation—whether the model identifies wh...

**来源信息**
- **来源**：Apple Machine Learning Research（RSS）
- **分类**：论文
- **发布时间**：2026-06-23 08:00（北京时间）
- **原文**：[打开原文](https://machinelearning.apple.com/research/metric-dependent-annotation-saturation)