**Hierarchical Residuals: A Minimalist Decoupling of Residual Width from Compute

**Hierarchical Residuals: A Minimalist Decoupling of Residual Width from Compute Width**

> Residual connections underpin Transformers, but they implicitly tie the residual width carried through depth to the compute width used inside each layer. We introduce Hierarchical Residuals (HiRes), a minimalist architectural change that decouples these dimensions, widening the residual state while keeping per-layer compute width fixed. HiRes groups layers into blocks: standard d-dimensional PreNorm layers operate in...

**来源信息**
- **来源**：HuggingFace Daily Papers（社区热门论文）
- **分类**：ai-models
- **发布时间**：2026-11-09 08:00（北京时间）
- **原文**：[打开原文](https://huggingface.co/papers/2610.41711)