**Exploring FlashAttention-3/4 optimizations on RTX GPUs**

> Part 2 takes Part 1&\#x27;s twisted monoid down to the metal: a generic compiler moveset walks one attention kernel from 25959µs to 206µs on an RTX 5090 — on par with FlashAttention-2 — with a measured number for every rung, the optimizations measurement refused, and the reasons FA-3/4 cannot land on consumer silicon. ← Compiler July 8, 2026 · TUTORIALS · GPU Learning FlashAttention the Hard Way — Part 2 Efficient CUD...

**来源信息**
- **来源**：Hacker News：AI 热帖
- **分类**：论文
- **发布时间**：2026-07-28 22:30（北京时间）
- **原文**：[打开原文](https://riftstack.ai/research/learning-flashattention-the-hard-way-part-2)