Nvidia GH200
Compare prices for Nvidia GH200 across cloud providers
Dec. 12, 2024 (updated)
Introduced in 2023, the Nvidia GH200 features the Grace Hopper architecture, which combines CPU and GPU technologies for large-scale AI and HPC workloads. Nvidia emphasizes its suitability for applications requiring extreme scalability and unified memory for efficient processing.
| Provider | GPUs | VRAM | vCPUs | RAM | Price/h | |
|---|---|---|---|---|---|---|
Lambda Labs
|
1x GH200 | 96GB | 64 | 432GB | $3.19 | Source |
Vultr
|
1x GH200 | 96GB | 72 | 480GB | $3.32 | Source |
Civo
|
1x GH200 | 96GB | -- | -- | On Request | Source |
Note: Prices are subject to change and may vary by region and other factors not listed here. For some GPUs, I include links to Shadeform (the sponsor) so you can check if they're available right now. I don’t earn a commission when you click on these links, but their monthly sponsorship helps me keep the site running.
Nvidia GH200 specs
| Feature | GH200 | GH200 NVL2 |
|---|---|---|
| CPU core count | 72 Arm Neoverse V2 cores | 144 Arm Neoverse V2 cores |
| L1 cache | 64KB i-cache + 64KB d-cache | 64KB i-cache + 64KB d-cache |
| L2 cache | 1MB per core | 1MB per core |
| L3 cache | 114MB | 228MB |
| Base frequency | all-core single instruction, multiple data (SIMD) frequency | 3.1GHz | 3.0GHz |
| LPDDR5X size | 480GB, 120GB, 240GB | 960GB, 240GB, 480GB |
| Memory bandwidth | Up to 384GB/s, Up to 512GB/s | Up to 768GB/s, Up to 1024GB/s |
| PCIe links | Up to 4x PCIe x16 (Gen5) | Up to 8x PCIe x16 (Gen5) |
| FP64 | 34 teraFLOPS | 68 teraFLOPS |
| FP64 Tensor Core | 67 teraFLOPS | 134 teraFLOPS |
| FP32 | 67 teraFLOPS | 134 teraFLOPS |
| TF32 Tensor Core | 989 teraFLOPS*, 494 teraFLOPS | 1,979 teraFLOPS*, 990 teraFLOPS |
| BFLOAT16 Tensor Core | 1,979 teraFLOPS*, 990 teraFLOPS | 3,958 teraFLOPS*, 1,979 teraFLOPS |
| FP16 Tensor Core | 1,979 teraFLOPS*, 990 teraFLOPS | 3,958 teraFLOPS*, 1,979 teraFLOPS |
| FP8 Tensor Core | 3,958 teraFLOPS*, 1,979 teraFLOPS | 7,916 teraFLOPS*, 3,958 teraFLOPS |
| INT8 Tensor Core | 3,958 TOPS*, 1,979 TOPS | 7,916 TOPS*, 3,958 TOPS |
| High-bandwidth memory (HBM) size | 96GB HBM3 | 144GB HBM3e, Up to 288GB HBM3e |
| Memory bandwidth | Up to 4TB/s, Up to 4.9TB/s | Up to 9.8TB/s |
| NVIDIA NVLink-C2C CPU-to-GPU bandwidth | 900 GB/s | 900 GB/s |
| Power | Configurable 450 to 1000W (Memory + CPU + GPU) | Configurable 900W to 2000W (Memory + CPU + GPU) |
| Thermal solution | Air cooled or liquid cooled | Air cooled or liquid cooled |
* with sparsity
Source: official Nvidia GH200 datasheet.