AMD CDNA 3 Architecture White Paper
AMD CDNA 3 Architecture White Paper #
AMD | 2023-12 | https://www.amd.com/content/dam/amd/en/documents/instinct-business-docs/white-papers/amd-cdna3-white-paper.pdf Category: hardware | Tags: gpu-architecture, matrix-core, chiplet, mi300x, mi325x, cdna3, 3d-stacking, infinity-cache Read: 2026-04-16
Core Contribution #
AMD革命性chiplet GPU架构CDNA 3，通过8个XCD(5nm)+4个IOD(6nm)的3D堆叠实现前所未有的计算密度，引入FP8/TF32/2:4稀疏，并以256MB Infinity Cache和192-256GB HBM3/3E显存成为LLM推理的有力竞争者。
Summary #
CDNA 3是GPU架构的范式转换——从传统monolithic/MCM演进为真正的异构chiplet 3D堆叠。MI300X由12个chiplet组成：8个XCD(Accelerator Complex Die, TSMC 5nm)承载计算单元，4个IOD(I/O Die, TSMC 6nm)承载Infinity Fabric控制器、HBM PHY、PCIe控制器和256MB Infinity Cache。XCD与IOD通过3D堆叠连接，实现了极高的通信带宽和能效。
304个CU(38/XCD)在FP8精度下达到2,614.9 TFLOPS，FP16达1,307.4 TFLOPS，首次支持OCP FP8标准(E4M3/E5M2)和TF32格式(490.3 TFLOPS)。2:4结构化稀疏支持可将FP16/BF16/FP8/INT8矩阵吞吐翻倍。
MI300X配备192GB HBM3(5.3TB/s)，MI325X升级至256GB HBM3E(6.0TB/s)。256MB Infinity Cache提供17.2TB/s带宽，大幅缓解内存墙。7条IF链路实现8-GPU全互联节点。MI300A APU变体将CPU和GPU统一封装并共享HBM3，开创新的异构计算范式。
Key Findings #
- 真正的chiplet架构: 8 XCD(5nm) + 4 IOD(6nm), 3D堆叠, 12个chiplet
- FP8: 2,614.9 TFLOPS, FP16: 1,307.4 TFLOPS, TF32: 490.3 TFLOPS
- 256MB Infinity Cache, 17.2TB/s带宽
- 192GB HBM3(MI300X) / 256GB HBM3E(MI325X)，显存容量领先同代NVIDIA
- 2:4结构化稀疏 + OCP FP8 (E4M3/E5M2) + TF32
- MI300A APU: CPU+GPU统一封装共享HBM3
- 7x IF链路实现8-GPU全互联，支持最多8分区
Key Tables #
Table 1: MI300X / MI325X 完整性能规格 #
| Computation | MI300X | MI325X |
|---|---|---|
| FP64 Vector | 81.7 TFLOPS | 81.7 TFLOPS |
| FP64 Matrix | 163.4 TFLOPS | 163.4 TFLOPS |
| FP32 Matrix | 163.4 TFLOPS | 163.4 TFLOPS |
| TF32 Matrix | 490.3 TFLOPS | 490.3 TFLOPS |
| FP16 Matrix | 1,307.4 TFLOPS | 1,307.4 TFLOPS |
| BF16 Matrix | 1,307.4 TFLOPS | 1,307.4 TFLOPS |
| FP8 Matrix | 2,614.9 TFLOPS | 2,614.9 TFLOPS |
| INT8 Matrix | 2,614.9 TOPS | 2,614.9 TOPS |
| 2:4 Sparsity | FP16/BF16/FP8/INT8 | FP16/BF16/FP8/INT8 |
| 显存 | 192 GB HBM3 | 256 GB HBM3E |
| 显存带宽 | 5.3 TB/s | 6.0 TB/s |
Takeaway: 计算性能MI300X/MI325X完全一致，MI325X的差异化在于HBM3E升级(+64GB, +0.7TB/s)
Table 2: MI300X vs MI250X 代际对比 #
| 指标 | MI300X | MI250X | 提升 |
|---|---|---|---|
| CU | 304 | 220 | 1.38x |
| FP64 Matrix | 163.4 TF | 95.7 TF | 1.71x |
| FP16 Matrix | 1,307 TF | 383 TF | 3.41x |
| FP8 Matrix | 2,615 TF | N/A | 新能力 |
| 显存 | 192 GB | 128 GB | 1.5x |
| 带宽 | 5.3 TB/s | 3.2 TB/s | 1.66x |
| Infinity Cache | 256 MB | N/A | 新能力 |
Takeaway: FP16 3.4x提升来自CU增加+per-CU矩阵吞吐提升，FP8和Infinity Cache为全新能力
Table 3: MI300X vs H100 SXM5 竞争对比 #
| 指标 | MI300X | H100 SXM5 | 对比 |
|---|---|---|---|
| FP64 Matrix | 163.4 TF | 66.9 TF | AMD 2.4x |
| FP16 TC | 1,307 TF | 989 TF | AMD +32% |
| FP8 TC | 2,615 TF | 1,979 TF | AMD +32% |
| 显存 | 192 GB | 80 GB | AMD 2.4x |
| 带宽 | 5.3 TB/s | 3.35 TB/s | AMD +58% |
| L2+LLC | 32+256 MB | 50 MB | AMD 5.7x |
Takeaway: MI300X在AI性能、显存容量、带宽上首次全面超越同代NVIDIA旗舰
Limitations #
- 无MXFP格式支持(sub-8bit仅有INT8)
- 跨XCD通信需经过IOD，延迟高于monolithic设计
- 无硬件Transformer Engine (NVIDIA H100有)
- LDS仅64KB/CU，限制了某些kernel的tile size
- 8-GPU全互联局限于节点内，缺乏NVLink Switch等跨节点方案
Infrastructure Impact #
- Algorithm: FP8训练成为可能，2:4稀疏可加速推理
- Kernel: Chiplet架构需要XCD-aware调度和数据布局优化
- Framework: ROCm + PyTorch/JAX支持MI300X，vLLM/SGLang开始适配
- LLM: 192-256GB显存支持单卡部署70B+模型，LLM推理竞争力显著
- System: Chiplet架构验证了GPU heterogeneous integration路线
Deep Analysis (hardware) #
Figure 1: MI300 Series 3D Package Chiplet Construction #
解读: AMD Instinct MI300系列的3D封装chiplet构造示意图。这是AMD GPU架构的范式转变——从monolithic/MCM进化到真正的异构chiplet 3D堆叠。底层是4个IOD(I/O Die, TSMC 6nm)，包含Infinity Cache和HBM3内存控制器。上层是8个XCD(Accelerator Complex Die, TSMC 5nm)，包含计算单元和L2 Cache。XCD通过3D堆叠垂直置于IOD之上。四周是8个HBM3 memory stacks(每stack 24GB, 总192GB)。IOD之间通过Infinity Fabric互联。MI300A APU变体将3个CCD(Zen4 CPU die)替代3个XCD，实现CPU+GPU统一封装。
Figure 2: MI300X and MI300A Block Diagrams #
解读: MI300X(纯GPU)和MI300A(APU)的block diagram对比。MI300X: 8个XCD + 4个IOD + 8个HBM3 stacks，304个CU(38/XCD)，192GB显存。MI300A: 6个XCD + 3个CCD(Zen4) + 4个IOD + 8个HBM3 stacks，228个CU + 24个Zen4核心，CPU和GPU共享128GB统一HBM3内存(消除了传统架构中CPU-GPU数据拷贝开销)。
Figure 3: XCD Block Diagram #
解读: 单个XCD(Accelerator Complex Die)的内部block diagram。包含：HWS(硬件调度器)、4个ACE(异步计算引擎)分别管理40个CU(产品启用38个，2个禁用用于良率)、4MB共享L2 Cache(16-way set assoc, 16通道)。L2作为XCD与Infinity Fabric网络的唯一接口——所有离开XCD的流量必须经过L2。这是AMD chiplet策略的核心构建块——小die+高良率，通过Infinity Fabric组合成大GPU。
Figure 4: CDNA3 Enhanced CU Architecture #
解读: CDNA 3 CU内部架构图。关键增强：(1)Matrix Core Unit新增FP8(E4M3/E5M2)和TF32支持，INT8吞吐提升6.8x vs CDNA2；(2)FP16/BF16吞吐翻倍至2048 FLOPS/clk/CU；(3)L1向量数据缓存容量翻倍至32KB(128B cache lines)，带宽翻倍；(4)指令缓存64KB(2x)，两CU共享；(5)支持2:4结构化稀疏(首次)，Matrix Core可跳过零值计算实现吞吐翻倍。CU内部保持Scalar+Vector+Matrix+Memory四条并行pipeline。
Figure 5: CDNA3 Memory Architecture #
解读: CDNA 3完整内存层次结构图——这是理解chiplet架构数据流的关键图。从上到下：CU内(L1 32KB, LDS 64KB) → XCD内(L2 4MB, 绿色通路 51.6 TB/s聚合) → 跨Infinity Fabric(黄色通路 17.2 TB/s聚合) → IOD内(Infinity Cache 256MB, 红色通路 5.3 TB/s聚合) → HBM3(192GB)。三种颜色的连线清晰标注了每一级的带宽。Infinity Cache是全新引入的memory-side cache——不参与coherency，仅缓存HBM内容，减少off-chip访问，提供17.2 TB/s的巨大带宽放大。
1. Architecture Overview #
Per-Chiplet Bandwidth Breakdown (逐chiplet带宽分拆) #
全芯片聚合带宽掩盖了单个chiplet的实际可用带宽。对kernel优化至关重要的是per-XCD视角:
| Level | Per-XCD | Per-IOD | Total (8 XCD / 4 IOD) | 推导 |
|---|---|---|---|---|
| L2 Read BW | 4.3 TB/s | 12.9 TB/s | 34.4 TB/s | \(128\ \text{B} \times 16\ \text{ch} \times 2.1\ \text{GHz}\) |
| L2 Write BW | 2.15 TB/s | 6.45 TB/s | 17.2 TB/s | \(64\ \text{B} \times 16\ \text{ch} \times 2.1\ \text{GHz}\) |
| L2 Total BW | 6.45 TB/s | 19.35 TB/s | 51.6 TB/s | Read + Write ✓ |
| LLC (Infinity Cache) BW | 2.15 TB/s | 4.3 TB/s | 17.2 TB/s | \(64\ \text{B} \times 16\ \text{ch} \times f_{\text{clk}}\) |
| HBM BW | 0.66 TB/s | 1.325 TB/s | 5.3 TB/s | \(1{,}024\ \text{b} \times 5.2\ \text{Gbps}\) |
关键洞察: 单个XCD只有0.66 TB/s的HBM带宽，对memory-bound kernel而言，数据局部性(L2 hit)至关重要——L2提供 \(6.5\times\) 的带宽放大(\(4.3\) vs \(0.66\ \text{TB/s}\))。跨XCD数据访问必须经过Infinity Fabric→LLC→HBM路径，带宽急剧下降。这解释了为什么CDNA3的NUMA-aware kernel优化如此重要。
- 产品: AMD Instinct MI300X / MI325X / MI300A / CDNA 3
- 工艺: XCD: TSMC 5nm (N5), IOD: TSMC 6nm (N6)
- 封装: Chiplet 3D堆叠 — 8 XCD + 4 IOD
- 目标市场: AI训练/推理 + HPC
2. Generational Delta (vs CDNA 2 / MI250X) #
新增: Chiplet 3D堆叠, Infinity Cache (256MB), FP8 (OCP E4M3/E5M2), TF32, 2:4结构化稀疏, GPU分区(最多8分区), MI300A APU
移除: 无
增强: CU 304(vs 220), 工艺5nm+6nm(vs 7nm), HBM3/3E(vs HBM2e), 显存192-256GB(vs 128GB), 带宽5.3-6.0TB/s(vs 3.2TB/s), 7x IF链路(vs 8x external)
3. Compute Unit Deep Dive #
- 每XCD: 38 CU (40 physical, 2 disabled for yield), 4MB L2 Cache
- 每CU: 1个Matrix Core Engine, 4个Vector单元(16-wide), Scalar单元
- L1 Data Cache: 32KB/CU (128B cache lines)
- LDS: \( C_{\text{LDS}} = 32 \times 512 \times \frac{32}{8} = 64\ \text{KB/CU} \)
- VRF: per-CU寄存器堆
- Wavefront: 64 work-items, \( C_{\text{wave}} = \frac{64}{16} = 4 \) cycles/SIMD
- Stream Processors: \( N_{\text{SP}} = 304 \times 64 = 19{,}456 \)
- Matrix Cores: \( N_{\text{MC}} = 304 \times 4 = 1{,}216 \)
- MFMA支持: FP64/FP32/TF32/FP16/BF16/FP8/INT8输入
- 2:4稀疏: FP16/BF16/FP8/INT8矩阵吞吐翻倍
4. Peak Performance Derivation #
\( f_{\text{boost}} \approx 2.1\ \text{GHz} \) (MI300X)
\[ \text{BF16 Matrix} = 2048\ \frac{\text{FLOPS}}{\text{clk}\cdot\text{CU}} \times 304\ \text{CU} \times 2.1\ \text{GHz} = 1{,}307\ \text{TFLOPS} \]
\[ \text{FP8 Matrix} = 4096\ \frac{\text{FLOPS}}{\text{clk}\cdot\text{CU}} \times 304\ \text{CU} \times 2.1\ \text{GHz} = 2{,}615\ \text{TFLOPS} \]
\[ \text{FP32 Vector} = 256\ \frac{\text{FLOPS}}{\text{clk}\cdot\text{CU}} \times 304\ \text{CU} \times 2.1\ \text{GHz} = 163.4\ \text{TFLOPS} \]
\[ \text{FP64 Matrix} = 256\ \frac{\text{FLOPS}}{\text{clk}\cdot\text{CU}} \times 304\ \text{CU} \times 2.1\ \text{GHz} = 163.4\ \text{TFLOPS} \]
| Data Type | FLOPS/clk/CU | Peak TFLOPS |
|---|---|---|
| FP64 Vector | 128 | 81.7 |
| FP64 Matrix | 256 | 163.4 |
| FP32 Matrix | 256 | 163.4 |
| TF32 Matrix | 1,024 | 490.3 |
| FP16 Matrix | 2,048 | 1,307.4 |
| BF16 Matrix | 2,048 | 1,307.4 |
| FP8 Matrix | 4,096 | 2,614.9 |
| INT8 Matrix | 4,096 | 2,614.9 TOPS |
5. Memory Subsystem #
\[ \text{BW}_{\text{HBM}} = \frac{1024\ \text{bits} \times 5.2\ \text{Gbps} \times 8\ \text{stacks}}{8} = 5{,}324\ \text{GB/s} \approx 5.3\ \text{TB/s} \]
- HBM3: 192GB, 5.3 TB/s (MI300X) / HBM3E: 256GB, 6.0 TB/s (MI325X)
- Infinity Cache: 256MB, 17.2 TB/s
- L2: \( 4\ \text{MB/XCD} \times 8 = 32\ \text{MB total} \)
- L1 Data Cache: 32KB/CU
- LDS: 64KB/CU
\[ \beta_{\text{FP8}} = \frac{2{,}615\ \text{TFLOPS}}{5.3\ \text{TB/s}} = 493\ \text{Ops/Byte} \]
\[ \beta_{\text{FP16}} = \frac{1{,}307\ \text{TFLOPS}}{5.3\ \text{TB/s}} = 247\ \text{Ops/Byte} \]
6. Interconnect #
\[ \text{BW}_{\text{IF,link}} = \frac{16\ \text{lanes} \times 32\ \text{Gbps} \times 2}{8} = 128\ \text{GB/s} \]
\[ \text{BW}_{\text{P2P,total}} = 128\ \text{GB/s} \times 7\ \text{links} = 896\ \text{GB/s} \]
- Infinity Fabric: 7x links, 全互联8-GPU节点
- PCIe Gen5: x16
- XCD-IOD: 3D堆叠互联
- GPU分区: 最多8分区 (1-8 XCD/分区)
7. Software & ISA #
- ROCm (持续演进)
- MFMA指令族: 新增FP8/TF32矩阵指令, 2:4稀疏指令
- OCP FP8标准: E4M3/E5M2双变体
- HIP + LLVM编译器
- vLLM/SGLang推理框架支持