Cloud GPU Cost Comparison

How much does it cost to run a GPU in the cloud?

July 11, 2024 (updated) by @anthonynsimon

Here's an overview of the different GPU models and their price range across various cloud providers:

GPU Model
Min. Price (per hour)
Available At
Nvidia RTX4000 $0.38 → Hetzner Hetzner, Paperspace
AMD 7900XTX $0.39 → DataCrunch DataCrunch
Nvidia T4G $0.42 → AWS AWS
Nvidia M4000 $0.45 → Paperspace Paperspace
Nvidia RTX6000 $0.50 → Lambda Labs DataCrunch, Lambda Labs, Linode
Nvidia P4000 $0.51 → Paperspace Paperspace
Nvidia T4 $0.53 → Azure Replicate, Azure
Nvidia V100 $0.62 → DataCrunch AWS, Azure, OVH, DataCrunch, Paperspace, Lambda Labs
Nvidia L4 $0.71 → GCP Scaleway, OVH, GCP
Nvidia A10G $0.75 → Lambda Labs AWS, Lambda Labs
Nvidia M60 $0.75 → AWS AWS, Azure
Nvidia A4000 $0.76 → Paperspace Paperspace
Nvidia P5000 $0.78 → Paperspace Paperspace
Nvidia A6000 $0.80 → Lambda Labs DataCrunch, Paperspace, Lambda Labs
Nvidia RTX5000 $0.82 → Paperspace Paperspace
Nvidia V100S $0.88 → OVH OVH
Nvidia K80 $0.90 → AWS AWS, Azure
Nvidia RTX3070 $1.07 → Scaleway Scaleway
Nvidia P6000 $1.10 → Paperspace Paperspace
Nvidia A100 $1.29 → DataCrunch Fly.io, AWS, Azure, GCP, OVH, Civo, Replicate, DataCrunch, Paperspace, Lambda Labs
Nvidia P100 $1.35 → Scaleway Azure, Scaleway
Nvidia L40S $1.36 → DataCrunch Scaleway, Fly.io, OVH, Civo, DataCrunch
Nvidia A5000 $1.38 → Paperspace Paperspace
Nvidia A10 $1.50 → Fly.io Fly.io
Nvidia A40 $2.07 → Replicate Replicate
Nvidia P40 $2.07 → Azure Azure
Nvidia H100 $2.49 → Lambda Labs Scaleway, AWS, OVH, Civo, DataCrunch, Paperspace, Lambda Labs
AMD MI300X On Request → TensorWave TensorWave
Nvidia GH100 On Request → Civo Civo

Please note that these cost estimates are based on the on-demand pricing for the region closest to N. Virginia (USA) or Frankfurt (Germany). Prices are in either USD or EUR, are subject to change, and may vary by region and other factors not listed here.

imgReplicate

Name GPUs Total GPU Memory vCPUs RAM Price (hour)
Nvidia T4 1x T4 16GB 4 16GB $0.81
Nvidia A40 (Small) 1x A40 48GB 4 16GB $2.07
Nvidia A40 (Large) 1x A40 48GB 10 72GB $2.61
Nvidia A100 (40GB) 1x A100 40GB 10 72GB $4.14
Nvidia A100 (80GB) 1x A100 80GB 10 144GB $5.04
2x Nvidia A40 (Large) 2x A40 96GB 20 144GB $5.22
2x Nvidia A100 (40GB) 2x A100 80GB 20 144GB $8.28
2x Nvidia A100 (80GB) 2x A100 160GB 20 288GB $10.08
4x Nvidia A40 (Large) 4x A40 192GB 40 288GB $10.44
4x Nvidia A100 (40GB) 4x A100 160GB 40 288GB $16.56
4x Nvidia A100 (80GB) 4x A100 320GB 40 576GB $20.16
8x Nvidia A40 (Large) 8x A40 384GB 48 680GB $20.88
8x Nvidia A100 (80GB) 8x A100 640GB 80 960GB $40.32

You can find more on Replicate GPU instances here.

imgAmazon Web Services

Name GPUs Total GPU Memory vCPUs RAM Price (hour)
g5g.xlarge 1x T4G 16GB 4 16GB $0.42
g4dn.xlarge 1x T4G 16GB 4 16GB $0.53
g5g.2xlarge 1x T4G 16GB 8 16GB $0.56
g4dn.2xlarge 1x T4G 16GB 8 32GB $0.75
g3s.xlarge 1x M60 8GB 4 30GB $0.75
g5g.4xlarge 1x T4G 16GB 16 16GB $0.83
p2.xlarge 1x K80 12GB 4 61GB $0.90
g5.xlarge 1x A10G 24GB 4 16GB $1.01
g3.4xlarge 1x M60 8GB 16 122GB $1.14
g4dn.4xlarge 1x T4G 16GB 16 64GB $1.20
g5.2xlarge 1x A10G 24GB 8 32GB $1.21
g5g.8xlarge 1x T4G 16GB 32 16GB $1.37
g5.4xlarge 1x A10G 24GB 16 64GB $1.62
g4dn.8xlarge 1x T4G 16GB 32 128GB $2.18
g3.8xlarge 2x M60 16GB 32 244GB $2.28
g5.8xlarge 1x A10G 24GB 32 128GB $2.45
g5g.16xlarge 2x T4G 32GB 64 32GB $2.74
p3.2xlarge 1x V100 16GB 8 61GB $3.06
g4dn.12xlarge 4x T4G 64GB 48 192GB $3.91
g5.12xlarge 4x A10G 96GB 48 192GB $4.10
g4dn.16xlarge 1x T4G 16GB 64 256GB $4.35
g3.16xlarge 4x M60 32GB 64 488GB $4.56
g5.16xlarge 2x A10G 48GB 64 256GB $5.67
p2.8xlarge 8x K80 96GB 32 488GB $7.20
g4dn.metal 8x T4G 128GB 96 384GB $7.82
g5.24xlarge 4x A10G 96GB 96 384GB $8.14
p3.8xlarge 4x V100 64GB 32 244GB $12.24
p2.16xlarge 16x K80 192GB 64 732GB $14.40
g5.48xlarge 8x A10G 192GB 192 768GB $16.29
p3.16xlarge 8x V100 128GB 64 488GB $24.48
p4d.24xlarge 8x A100 320GB 96 1152GB $32.77
p5.48xlarge 8x H100 640GB 192 2048GB $98.32

You can find more on AWS GPU instances here.

imgGoogle Cloud

Name GPUs Total GPU Memory vCPUs RAM Price (hour)
g2-standard-4 1x L4 24GB 4 16GB $0.71
g2-standard-8 1x L4 24GB 8 32GB $0.85
g2-standard-12 1x L4 24GB 12 48GB $1.00
g2-standard-16 1x L4 24GB 16 64GB $1.15
g2-standard-32 1x L4 24GB 32 128GB $1.73
g2-standard-24 2x L4 48GB 24 96GB $2.00
a2-highgpu-1g 1x A100 40GB 12 85GB $3.67
g2-standard-48 4x L4 96GB 48 192GB $4.00
a2-highgpu-2g 2x A100 80GB 24 170GB $7.35
g2-standard-96 8x L4 192GB 96 384GB $8.00
a2-highgpu-4g 4x A100 160GB 48 340GB $14.69
a2-highgpu-8g 8x A100 320GB 96 680GB $29.39
a2-megagpu-16g 16x A100 640GB 96 1360GB $55.74

You can find more on GCP GPU instances here.

imgMicrosoft Azure

Name GPUs Total GPU Memory vCPUs RAM Price (hour)
NC4as T4 v3 1x T4 16GB 4 28GB $0.53
NC8as T4 v3 1x T4 16GB 8 56GB $0.75
NC6 1x K80 12GB 6 56GB $0.90
NV6 1x M60 8GB 6 56GB $1.14
NV12s v3 1x M60 8GB 12 112GB $1.14
NC16as T4 v3 1x T4 16GB 16 110GB $1.20
NC12 2x K80 24GB 12 112GB $1.80
NC6s v2 1x P100 16GB 6 112GB $2.07
ND6s 1x P40 24GB 6 112GB $2.07
NV12 2x M60 16GB 12 112GB $2.28
NV24s v3 2x M60 16GB 24 224GB $2.28
NC6s v3 1x V100 16GB 6 112GB $3.06
NC24 4x K80 48GB 24 224GB $3.60
NC24r 4x K80 48GB 24 224GB $3.96
NC12s v2 2x P100 32GB 12 224GB $4.14
ND12s 2x P40 48GB 12 224GB $4.14
NC64as T4 v3 4x T4 64GB 64 440GB $4.35
NV48s v3 4x M60 32GB 48 448GB $4.56
NV24 4x M60 32GB 24 224GB $4.56
NC12s v3 2x V100 32GB 12 224GB $6.12
NC24s v2 4x P100 64GB 24 448GB $8.28
ND24s 4x P40 96GB 24 448GB $8.28
NC24rs v2 4x P100 64GB 24 448GB $9.11
ND24rs 4x P40 96GB 24 448GB $9.11
NC24s v3 4x V100 64GB 24 448GB $12.24
NC24rs v3 4x V100 64GB 24 448GB $13.46
ND40rs v2 8x V100 256GB 40 672GB $22.03
ND96asr A100 v4 8x A100 320GB 96 900GB $27.20

imgLinode

Name GPUs Total GPU Memory vCPUs RAM Price (hour)
GPU1 1x RTX6000 24GB 8 32GB $1.50
GPU2 2x RTX6000 48GB 16 64GB $3.00
GPU3 3x RTX6000 72GB 20 96GB $4.50
GPU4 4x RTX6000 96GB 24 128GB $6.00

You can find more on Linode GPU instances here.

imgPaperspace

Name GPUs Total GPU Memory vCPUs RAM Price (hour)
GPU+ (M4000) 1x M4000 8GB 8 30GB $0.45
P4000 1x P4000 8GB 8 30GB $0.51
RTX4000 1x RTX4000 24GB 8 30GB $0.56
A4000 1x A4000 16GB 8 45GB $0.76
P5000 1x P5000 16GB 8 30GB $0.78
RTX5000 1x RTX5000 16GB 8 30GB $0.82
P6000 1x P6000 24GB 8 30GB $1.10
A5000 1x A5000 24GB 8 45GB $1.38
A6000 1x A6000 48GB 8 45GB $1.89
V100 1x V100 16GB 8 30GB $2.30
V100-32G 1x V100 32GB 8 30GB $2.30
A100 1x A100 40GB 12 90GB $3.09
A100-80G 1x A100 80GB 12 90GB $3.19
H100 1x H100 80GB 20 256GB $5.95

You can find more on Paperspace GPU instances here.

imgScaleway

Name GPUs Total GPU Memory vCPUs RAM Price (hour)
L4-1-24G 1x L4 24GB 8 48GB $0.82
GPU-3070 1x RTX3070 8GB 8 16GB $1.07
Render-S 1x P100 16GB 10 42GB $1.35
L40S-1-48G 1x L40S 48GB 8 96GB $1.52
L4-2-24G 2x L4 48GB 16 96GB $1.63
H100-1-80G 1x H100 80GB 24 240GB $2.74
L40S-2-48G 2x L40S 96GB 16 192GB $3.05
L4-4-24G 4x L4 96GB 32 192GB $3.27
H100-2-80G 2x H100 160GB 48 480GB $5.49
L40S-4-48G 4x L40S 192GB 32 384GB $6.10
L4-8-24G 8x L4 192GB 64 384GB $6.53
L40S-8-48G 8x L40S 384GB 64 768GB $12.20

You can find more on Scaleway GPU instances here.

imgLambda Labs

Name GPUs Total GPU Memory vCPUs RAM Price (hour)
1x RTX6000 1x RTX6000 24GB 14 46GB $0.50
1x A10 1x A10G 24GB 30 200GB $0.75
1x A6000 1x A6000 48GB 14 100GB $0.80
1x A100 1x A100 40GB 30 200GB $1.29
2x A6000 2x A6000 96GB 28 200GB $1.60
1x H100 1x H100 80GB 26 200GB $2.49
2x A100 2x A100 80GB 60 400GB $2.58
4x A6000 4x A6000 192GB 56 400GB $3.20
8x V100 8x V100 128GB 92 448GB $4.40
4x A100 4x A100 160GB 120 800GB $5.16
8x A100 8x A100 320GB 124 1800GB $10.32
8x H100 8x H100 640GB 208 1800GB $27.92

You can find more on Lambda Labs GPU instances here.

imgOVHcloud

Name GPUs Total GPU Memory vCPUs RAM Price (hour)
t1-le-45 1x V100 16GB 8 45GB $0.77
t2-le-45 1x V100S 32GB 15 45GB $0.88
l4-90 1x L4 24GB 22 90GB $1.00
t1-le-90 2x V100 32GB 16 90GB $1.55
t2-le-90 2x V100S 64GB 30 90GB $1.76
l40s-90 1x L40S 48GB 15 90GB $1.80
t1-45 1x V100 16GB 8 45GB $1.97
l4-180 2x L4 48GB 45 180GB $2.00
t2-45 1x V100S 32GB 15 45GB $2.19
h100-380 1x H100 80GB 30 380GB $2.99
a100-180 1x A100 80GB 15 180GB $3.07
t1-le-180 4x V100 64GB 32 180GB $3.10
t2-le-180 4x V100S 128GB 60 180GB $3.53
l40s-180 2x L40S 96GB 30 180GB $3.60
t1-90 2x V100 32GB 18 90GB $3.94
l4-360 4x L4 96GB 90 360GB $4.00
t2-90 2x V100S 64GB 30 90GB $4.38
h100-760 2x H100 160GB 60 760GB $5.98
a100-360 2x A100 160GB 30 360GB $6.15
l40s-360 4x L40S 192GB 60 360GB $7.20
t1-180 4x V100 64GB 36 180GB $7.89
t2-180 4x V100S 128GB 60 180GB $8.76
h100-1520 4x H100 320GB 120 1520GB $11.97
a100-720 4x A100 320GB 60 720GB $12.29

You can find more on OVHcloud GPU instances here.

imgFly.io

Fly.io charges for GPU usage in addition to the regular compute costs. Here's a breakdown of their on-demand GPU pricing:

Name GPUs Total GPU Memory vCPUs RAM Price (hour)
A10 1x A10 24GB -- -- $1.50
A100 40G PCIe 1x A100 40GB -- -- $2.50
L40S 1x L40S 48GB -- -- $2.50
A100 80G SXM 1x A100 80GB -- -- $3.50

You can find more on Fly.io GPU instances here.

imgCivo Cloud

Here's a breakdown of Civo Cloud's GPU pricing:

Name GPUs Total GPU Memory vCPUs RAM Price (hour)
GH200 1x GH100 96GB -- -- On Request
H100 1x H100 80GB -- -- On Request
A100-40 Small 1x A100 40GB 8 64GB $1.79
L40S Medium 1x L40S 48GB 8 64GB $1.79
A100-80 Small 1x A100 80GB 12 128GB $2.38
A100-40 Medium 2x A100 80GB 16 128GB $3.57
L40S Medium (2x) 2x L40S 96GB 16 128GB $3.57
A100-80 Medium 2x A100 160GB 24 256GB $4.76
A100-40 Large 4x A100 160GB 32 255GB $7.14
L40S Large 4x L40S 192GB 32 255GB $7.14
A100-80 Large 4x A100 320GB 48 512GB $9.52
A100-40 Extra Large 8x A100 320GB 64 512GB $14.29
L40S Extra Large 8x L40S 384GB 64 512GB $14.29
A100-80 Extra Large 8x A100 640GB 96 1024GB $19.05

You can find more on Civo Cloud GPU instances here.

imgHetzner

Hetzner does not currently offer on-demand GPU instances, but they do offer a dedicated server with an Nvidia RTX4000 GPU.

In addition to the hourly rate, there's a €94.01 setup fee (one time). Here are the specs:

Name GPUs Total GPU Memory vCPUs RAM Price (hour)
GEX44 1x RTX4000 20GB 14 64GB $0.38

You can find more on Hetzner GPU servers here.

imgTensorWave

TensorWave is a cloud provider leveraging AMD's Instinct™ MI300X accelerators.

When compared to Nvidia's H100, these GPUs come with higher memory capacity, bandwidth and processing power at a lower total cost of ownership.

While their pricing is not yet publicly available, you can request a quote on their website.

Name GPUs Total GPU Memory vCPUs RAM Price (hour)
AMD MI300X 8x MI300X 1536GB -- -- On Request

imgDataCrunch

DataCrunch offers GPU instances with the following base configurations which can be scaled by 2x, 4x, or 8x:

Name GPUs Total GPU Memory vCPUs RAM Price (hour)
1RX7900XTX.10V 1x 7900XTX 24GB 10 40GB $0.39
1V100.6V 1x V100 16GB 6 23GB $0.62
1A6000.10V 1x A6000 48GB 10 60GB $1.01
1RTX6000ADA.10V 1x RTX6000 48GB 10 60GB $1.27
1A100.40S.22V 1x A100 40GB 22 120GB $1.29
1L40S.20V 1x L40S 48GB 20 60GB $1.36
1A100.80S.22V 1x A100 80GB 22 120GB $1.75
1H100.80S.30V 1x H100 80GB 30 120GB $3.17

You can find more on DataCrunch GPU instances here.


Tip: You can use our website to compare cloud providers to find which one is the best fit for your use case.