Nvidia GH200
Last updated on Dec. 3, 2024
Price Comparison
| Provider | GPUs | VRAM | vCPUs | RAM | Price/h | |
|---|---|---|---|---|---|---|
Vultr
|
1x GH200 | 96GB | 72 | 480GB | $3.32 | Source |
Civo
|
1x GH200 | 96GB | -- | -- | On Request | Source |
Lambda Labs
|
1x GH200 | 96GB | 72 | -- | On Request | Source |
Note: Prices are subject to change, and may vary by region and other factors not listed here. I included links to launch GPUs using Shadeform (sponsor) so you can check if they're available right now. There are no fees to use their service.
Specifications
| Feature | GH200 | GH200 NVL2 |
|---|---|---|
| CPU core count | 72 Arm Neoverse V2 cores | 144 Arm Neoverse V2 cores |
| L1 cache | 64KB i-cache + 64KB d-cache | 64KB i-cache + 64KB d-cache |
| L2 cache | 1MB per core | 1MB per core |
| L3 cache | 114MB | 228MB |
| Base frequency | all-core single instruction, multiple data (SIMD) frequency | 3.1GHz | 3.0GHz |
| LPDDR5X size | 480GB, 120GB, 240GB | 960GB, 240GB, 480GB |
| Memory bandwidth | Up to 384GB/s, Up to 512GB/s | Up to 768GB/s, Up to 1024GB/s |
| PCIe links | Up to 4x PCIe x16 (Gen5) | Up to 8x PCIe x16 (Gen5) |
| FP64 | 34 teraFLOPS | 68 teraFLOPS |
| FP64 Tensor Core | 67 teraFLOPS | 134 teraFLOPS |
| FP32 | 67 teraFLOPS | 134 teraFLOPS |
| TF32 Tensor Core | 989 teraFLOPS*, 494 teraFLOPS | 1,979 teraFLOPS*, 990 teraFLOPS |
| BFLOAT16 Tensor Core | 1,979 teraFLOPS*, 990 teraFLOPS | 3,958 teraFLOPS*, 1,979 teraFLOPS |
| FP16 Tensor Core | 1,979 teraFLOPS*, 990 teraFLOPS | 3,958 teraFLOPS*, 1,979 teraFLOPS |
| FP8 Tensor Core | 3,958 teraFLOPS*, 1,979 teraFLOPS | 7,916 teraFLOPS*, 3,958 teraFLOPS |
| INT8 Tensor Core | 3,958 TOPS*, 1,979 TOPS | 7,916 TOPS*, 3,958 TOPS |
| High-bandwidth memory (HBM) size | 96GB HBM3 | 144GB HBM3e, Up to 288GB HBM3e |
| Memory bandwidth | Up to 4TB/s, Up to 4.9TB/s | Up to 9.8TB/s |
| NVIDIA NVLink-C2C CPU-to-GPU bandwidth | 900 GB/s | 900 GB/s |
| Power | Configurable 450 to 1000W (Memory + CPU + GPU) | Configurable 900W to 2000W (Memory + CPU + GPU) |
| Thermal solution | Air cooled or liquid cooled | Air cooled or liquid cooled |
* with sparsity
Source: official Nvidia GH200 datasheet.