Nvidia GH200

Compare prices for Nvidia GH200 across cloud providers

Dec. 12, 2024 (updated)

Introduced in 2023, the Nvidia GH200 features the Grace Hopper architecture, which combines CPU and GPU technologies for large-scale AI and HPC workloads. Nvidia emphasizes its suitability for applications requiring extreme scalability and unified memory for efficient processing.

Provider GPUs VRAM vCPUs RAM Price/h
Lambda Labs logo Lambda Labs 1x GH200 96GB 64 432GB $3.19 Source
Vultr logo Vultr 1x GH200 96GB 72 480GB $3.32 Source
Civo logo Civo 1x GH200 96GB -- -- On Request Source

Note: Prices are subject to change and may vary by region and other factors not listed here. For some GPUs, I include links to Shadeform (the sponsor) so you can check if they're available right now. I don’t earn a commission when you click on these links, but their monthly sponsorship helps me keep the site running.

Nvidia GH200 specs

Feature GH200 GH200 NVL2
CPU core count 72 Arm Neoverse V2 cores 144 Arm Neoverse V2 cores
L1 cache 64KB i-cache + 64KB d-cache 64KB i-cache + 64KB d-cache
L2 cache 1MB per core 1MB per core
L3 cache 114MB 228MB
Base frequency | all-core single instruction, multiple data (SIMD) frequency 3.1GHz 3.0GHz
LPDDR5X size 480GB, 120GB, 240GB 960GB, 240GB, 480GB
Memory bandwidth Up to 384GB/s, Up to 512GB/s Up to 768GB/s, Up to 1024GB/s
PCIe links Up to 4x PCIe x16 (Gen5) Up to 8x PCIe x16 (Gen5)
FP64 34 teraFLOPS 68 teraFLOPS
FP64 Tensor Core 67 teraFLOPS 134 teraFLOPS
FP32 67 teraFLOPS 134 teraFLOPS
TF32 Tensor Core 989 teraFLOPS*, 494 teraFLOPS 1,979 teraFLOPS*, 990 teraFLOPS
BFLOAT16 Tensor Core 1,979 teraFLOPS*, 990 teraFLOPS 3,958 teraFLOPS*, 1,979 teraFLOPS
FP16 Tensor Core 1,979 teraFLOPS*, 990 teraFLOPS 3,958 teraFLOPS*, 1,979 teraFLOPS
FP8 Tensor Core 3,958 teraFLOPS*, 1,979 teraFLOPS 7,916 teraFLOPS*, 3,958 teraFLOPS
INT8 Tensor Core 3,958 TOPS*, 1,979 TOPS 7,916 TOPS*, 3,958 TOPS
High-bandwidth memory (HBM) size 96GB HBM3 144GB HBM3e, Up to 288GB HBM3e
Memory bandwidth Up to 4TB/s, Up to 4.9TB/s Up to 9.8TB/s
NVIDIA NVLink-C2C CPU-to-GPU bandwidth 900 GB/s 900 GB/s
Power Configurable 450 to 1000W (Memory + CPU + GPU) Configurable 900W to 2000W (Memory + CPU + GPU)
Thermal solution Air cooled or liquid cooled Air cooled or liquid cooled

* with sparsity

Source: official Nvidia GH200 datasheet.