AMD has officially unveiled its Instinct MI300 APUs which combine Zen 4 CPU cores with CDNA 3 GPU cores with up to 153 Billion transistors & 192 GB HBM3 memory.
AMD Leads The Industry In Offering The First Exascale Instinct MI300 APU: Up To 24 Zen 4 Cores, GPU-Only MI300X With Up To 192 GB HBM3 Memory
All three chipmakers, AMD, Intel, and NVIDIA, are currently in the race to offer the world's first CPU and CPU combo. Intel's offering was going to be Falcon Shores which was going to combine an x86 CPU with a leading-edge GPU but those plans didn't come to fruition and the Chipzilla has delayed the project for the foreseeable future.
Meanwhile, NVIDIA's answer comes in the form of the Grace and Hopper Superchips which combine the Grace ARM CPU package with the Hopper H200 GPU on the same board but that's still a far cry from AMD's chiplet-based and on-package integration of Zen 4 CPU and CDNA 3 GPU cores.
Introducing the World’s Most Advanced Accelerator for Generative AI.
AMD revealed new details of the AMD Instinct MI300 Series accelerator family, including the introduction of the AMD Instinct MI300X accelerator, the world’s most advanced accelerator for generative AI. The MI300X is based on the next-gen AMD CDNA 3 accelerator architecture and supports up to 192 GB of HBM3 memory to provide the compute and memory efficiency needed for large language model training and inference for generative AI workloads.
With the large memory of AMD Instinct MI300X, customers can now fit large language models such as Falcon-40, a 40B parameter model on a single, MI300X accelerator5. AMD also introduced the AMD Instinct™ Platform, which brings together eight MI300X accelerators into an industry-standard design for the ultimate solution for AI inference and training. The MI300X is sampling to key customers starting in Q3. AMD also announced that the AMD Instinct MI300A, the world’s first APU Accelerator for HPC and AI workloads, is now sampling to customers.
So as of right now, AMD is truly taking charge of offering the first integrated x86 CPU and GPU package in the form of the Instinct MI300 APU. The same APU is going to be the lead product to secure AMD's strategic dominance within the AI segment. The AMD Instinct MI300 APU is going to power the El Capitan Supercomputer, delivering over 2 Exaflops of double precision horsepower. Rounding up some of the highlighted features of the AMD Instinct MI300 APUs, we have:
- First Integrated CPU+GPU Package
- Aiming Exascale Supercomputer Market
- AMD MI300A (Integrated CPU + GPU)
- AMD MI300X (GPU Only)
- 153 Billion Transistors
- Up To 24 Zen 4 Cores
- CDNA 3 GPU Architecture
- Up To 192 GB HBM3 Memory
- Up To 8 Chiplets + 8 Memory Stacks (5nm + 6nm process)
- MI300A Sampling Now, MI300X Sampling Next Quarter, Ramp In Q4 2023
AMD will be utilizing both 5nm and 6nm process nodes for its Instinct MI300 'CDNA 3' APUs. The chip will be outfitted with the next generation of Infinity Cache and feature the 4th Gen Infinity architecture which enables CXL 3.0 ecosystem support.
The Instinct MI300 accelerator will rock a unified memory APU architecture and new Math Formats, allowing for a 5x performance per watt uplift over CDNA 2 which is massive. AMD is also projecting over 8x the AI performance versus the CDNA 2-based Instinct MI250X accelerators. The CDNA 3 GPU's UMAA will connect the CPU and GPU to a unified HBM memory package, eliminating redundant memory copies while delivering low TCO.
The AMD Instinct MI300 will be available in two flavors, the MI300A, and the MI300X. The APU will encompass several 3D chiplet packages, all combining to house an insane 153 Billion transistors. Those transistors include various core IPs, memory interfaces, interconnects, and much more.
The CDNA 3 architecture is the fundamental DNA of the Instinct MI300 but the APU also comes with up to 304 CDNA 4 Compute Units for a total of 24,576 cores, up to 24 Zen 4, and up to 192 GB of the next-generation HBM3 memory running in 8192-bit wide bus config. Following is how these configurations are divided in terms of GPU CUs, CPU Cores, and memory:
- MI300A - 6 XCDs (Up To 228 CUs), 3 CCDs (Up To 24 Zen 4 Cores), 8 HBM3 Stacks (128 GB)
- MI300X - 8 XCDs (Up To 304 CUs), 0 CCDs (Up To 0 Zen 4 Cores), 8 HBM3 Stacks (192 GB)
As you can see, AMD's Instinct MI300 APUs will come in various configurations with CPU-only, GPU-only, and CPU+GPU integrated SKUs. The Instinct MI300A and Instinct MI300X will target the CPU / GPU workloads.
AMD's Instinct MI300 APU lineup will be compatible with a brand new socket known as the SH5, allowing for up to four such chips to be configured together using the latest Infinity Fabric inter-connect and new interconnect switches for faster bandwidth and inter-connectivity speeds. The server solution will come in the form of the AMD Instinct Platform that will comprise of 8 MI300 chips with up to 1.5 TB of HBM3 memory, all in an industry-standard design.
In the latest performance comparisons, AMD showcased that the Instinct Mi300 delivers an 8x boost in AI performance (TFLOPs) and a 5x AI performance per watt (TFLOPs/watt) boost over the Instinct MI250X. AMD's Instinct MI300A APU accelerators are sampling now and the MI300X will be sampling in Q3 2023. Both products are also expected to ramp up in Q4 of 2023.
