AI Data Center GPU Specifications
Comprehensive specifications for every major AI training and inference GPU. All data verified against official vendor datasheets. Covers NVIDIA Hopper and Blackwell, AMD CDNA 3, and Intel Gaudi architectures.
What Are the Best GPUs for AI Training in 2026?
| GPU | Arch | Memory | Bandwidth | FP16 TFLOPS | FP8 TFLOPS | TDP | Interconnect | Released |
|---|---|---|---|---|---|---|---|---|
| NVIDIA B200 Newest | Blackwell | 192 GB HBM3e | 8.0 TB/s | 4,500 | 9,000 | 1,000W | NVLink 5.0 (1,800 GB/s) | 2025 |
| NVIDIA H200 SXM | Hopper | 141 GB HBM3e | 4.8 TB/s | 989 | 1,979 | 700W | NVLink 4.0 (900 GB/s) | 2024 |
| NVIDIA H100 SXM Most Available | Hopper | 80 GB HBM3 | 3.35 TB/s | 989 | 1,979 | 700W | NVLink 4.0 (900 GB/s) | 2023 |
| NVIDIA H100 PCIe | Hopper | 80 GB HBM3 | 2.0 TB/s | 756 | — | 350W | PCIe Gen 5.0 | 2023 |
| NVIDIA A100 SXM 80GB | Ampere | 80 GB HBM2e | 2.0 TB/s | 312 | — | 400W | NVLink 3.0 (600 GB/s) | 2020 |
| AMD MI300X | CDNA 3 | 192 GB HBM3 | 5.3 TB/s | 1,307 | 2,614 | 750W | Infinity Fabric (896 GB/s) | 2024 |
| Intel Gaudi 3 | Custom ASIC | 128 GB HBM2e | 3.7 TB/s | 1,835 | 3,670 | 900W | RoCE v2 (24×200GbE) | 2025 |
Data from NVIDIA datasheets, AMD MI300X, and Intel Gaudi 3.
NVIDIA Data Center GPU Specifications
What are NVIDIA H100 SXM specifications?
The NVIDIA H100 SXM delivers 989 TFLOPS FP16, 80 GB HBM3 at 3.35 TB/s, NVLink 4.0 at 900 GB/s, 700W TDP.
| Specification | H100 SXM | H100 PCIe |
|---|---|---|
| Memory | 80 GB HBM3 | 80 GB HBM3 |
| Memory Bandwidth | 3.35 TB/s | 2.0 TB/s |
| FP16 / BF16 | 989 TFLOPS | 756 TFLOPS |
| TDP | 700W | 350W |
| Interconnect | NVLink 4.0 (900 GB/s) | Optional NVLink Bridge |
| MSRP | ~$30,000 | ~$25,000 |
What are AMD MI300X specifications?
AMD MI300X has 192 GB HBM3 and 5.3 TB/s bandwidth. MI300X offers 2.4x more memory, 1.58x more bandwidth, and 1.32x more FP16 TFLOPS vs H100 SXM at ~$15,000 MSRP vs H100's ~$30,000.
| Specification | MI300X | H100 SXM |
|---|---|---|
| Architecture | CDNA 3 (TSMC 5nm+6nm) | Hopper (TSMC 4N) |
| GPU Memory | 192 GB HBM3 | 80 GB HBM3 |
| Memory Bandwidth | 5.3 TB/s | 3.35 TB/s |
| FP16 Performance | 1,307 TFLOPS | 989 TFLOPS |
| TDP | 750W | 700W |
| MSRP | ~$15,000 | ~$30,000 |
Frequently Asked Questions
How much memory does NVIDIA H100 have?
NVIDIA H100 SXM has 80 GB of HBM3 memory with 3.35 TB/s bandwidth. The H200 increases this to 141 GB HBM3e at 4.8 TB/s.
How does AMD MI300X compare to NVIDIA H100?
AMD MI300X vs H100 SXM: 2.4x more memory (192 vs 80 GB), 1.58x more bandwidth (5.3 vs 3.35 TB/s), 1.32x more FP16 TFLOPS (1,307 vs 989), at ~$15K vs ~$30K MSRP. MI300X excels at memory-bound workloads like large model inference.
What GPU is best for inference workloads?
For inference: Small models (≤13B): L40S or L4 — best cost-per-token. 70B models: single H200 (141 GB fits LLaMA 70B FP16) or 2× H100. 100B+ models: AMD MI300X (192 GB) or multi-GPU cluster.