**Improved Large Language Diffusion Models** > Modern large language models are

**Improved Large Language Diffusion Models**

> Modern large language models are predominantly trained with autoregressive factorization and causal attention. We present iLLaDA, an 8B masked diffusion language model trained from scratch with fully bidirectional attention. iLLaDA keeps the masked diffusion objective throughout pre-training and supervised fine-tuning (SFT), scaling pre-training to 12T tokens and fine-tuning on a 25B-token instruction corpus for 12 e...

**来源信息**
- **来源**：HuggingFace Daily Papers（社区热门论文）
- **分类**：论文
- **发布时间**：2026-06-25 08:00（北京时间）
- **原文**：[打开原文](https://huggingface.co/papers/2606.25331)