Best Value Cloud GPUs for LLMs: Memory vs Bandwidth
What it costs to hold a model vs what it costs to run it fast.
Aug. 17, 2026
Every widely listed cloud GPU priced two ways: per GB of VRAM, and per TB/s of memory bandwidth. Cross-provider medians for the week of August 17, 2026, across 26 datacenter and workstation cards.
This week's picks (week of August 17, 2026)
- Best value for holding big models: AMD MI325X. 256 GB at $0.0120 per GB-hour, 58% below the market median, and it is under the median per unit of bandwidth too at $0.51 per TB/s-hour; it runs ROCm rather than CUDA, so check that your workload ports.
- Best value for fast inference: AMD MI300X. 5.3 TB/s at $2.90 per hour works out to $0.55 per TB/s-hour, 45% under the median.
- Best budget card for fine-tuning and fits: Nvidia RTX A6000. $0.0116 per GB-hour and $0.72 per TB/s-hour, under the median on both axes, with 48 GB for $0.56 per hour.
- Priced above the median on both axes: Nvidia H100. The most widely listed card we track, at 41 providers in this week's snapshot, and at $0.0423 per GB-hour it is 49% above the median; the premium buys throughput and ecosystem maturity the chart does not measure.
- Cheap by the hour, expensive by the unit: Nvidia V100 and Nvidia L4. $0.97 and $0.88 per hour, but $0.0302 and $0.0365 per GB-hour and $1.07 and $2.92 per TB/s-hour, against medians of $0.0284 and $1.00.
Which cards fit your model
Start with the AI model, not a GPU card. The table prices the memory each one needs at this week's median of $0.0284 per GB-hour, then names the cards that actually fit it.
| Model | VRAM needed (Q4) | Cost at median $/GB-hr | Cheapest cards that fit |
|---|---|---|---|
| Qwen3-Coder 30B-A3B | ~18 GB | ~$0.52/hr | RTX 4000 Ada ($0.50/hr, tight), RTX A6000 ($0.56/hr) |
| Gemma 4 31B | ~18 GB | ~$0.52/hr | RTX 4000 Ada ($0.50/hr, tight), RTX A6000 ($0.56/hr) |
| Llama 3.3 70B | ~42 GB | ~$1.19/hr | RTX A6000 ($0.56/hr, tight), A40 ($0.74/hr, tight) |
| gpt-oss-120b | ~70 GB | ~$1.99/hr | A100 ($1.76/hr, tight), RTX PRO 6000 ($2.19/hr) |
| Qwen3 235B-A22B | ~141 GB | ~$4.00/hr | MI300X ($2.90/hr), MI325X ($3.07/hr) |
| GLM-5.2 | ~452 GB | ~$12.85/hr | 2× MI325X ($6.14/hr, tight), 4× MI300X ($11.60/hr) |
VRAM is parameters × bytes per weight at Q4, plus 20% for KV cache and runtime overhead. Only cards under the median per gigabyte are offered as fits; under 15% headroom is marked tight.
Read the last column, not the middle one. The median is what the market charges for that much memory; the cards are what you would actually rent, and the gap between them is the point of this page.
Even the biggest model we price fits in one 8-GPU node. Kimi K3 (2.8T) needs about 1680 GB at Q4: 8× MI325X at $24.56 an hour.
This prices holding a model, not serving it quickly. If your users are waiting on tokens, read the next section before renting the cheapest thing that fits.
Fit or speed: which price applies to you
The cards here run from $0.21 to $7.89 an hour, and from 16 GB to 288 GB of memory. That 18x spread in what a card holds makes the hourly price close to meaningless on its own, so divide it by the spec your workload actually consumes:
- Cost per GB-hour is what you pay to hold a model of a given size. Shop on this if you run mixture-of-experts models, which keep every parameter in memory but activate only a slice per token, so capacity climbed with parameter count while bandwidth demand did not follow. It is why high-capacity cards lead the ranking.
- Cost per TB/s-hour is what you pay to stream weights, which sets decode speed on memory-bound inference. Shop on this if you run dense models or serve interactive traffic, where every token streams every weight and latency tracks bandwidth almost linearly.
The V100 shows why the hourly price misleads: $0.0302 per GB-hour and $1.08 per TB/s-hour, against medians of $0.0284 and $1.00. Cheap by the hour, expensive at everything it does.
Is the H100 good value?
For capacity the H100 is 49% above median: $0.0423 per GB-hour against a $0.0284 median. For speed it is roughly fair: $1.01 per TB/s-hour against $1.00. It is the most listed card we track, at 41 providers in this week's snapshot. If your model merely has to fit, the MI325X holds 256 GB at $0.0120 per GB-hour, 72% less per gigabyte.
Blackwell is cheaper than it looks
The B300 costs $7.89 per hour and still lands below the median on both axes: $0.0274 per GB-hour and $0.99 per TB/s-hour. An hourly price that looks out of reach is not the same as poor value.
Same specs, different price
The L40 and the L40S both offer 48 GB at 0.86 TB/s. The L40 rents for $1.00 per hour, the L40S for $1.50. Neither axis on this chart can tell them apart, so unless something the chart does not measure decides it for you, that is 33% off the same memory at the same speed.
What this does not tell you
Neither axis measures FP16 or FP8 throughput, NVLink, ECC, MIG support, provider reliability, or who is selling. A card can be the best value per gigabyte and still be wrong because your training run needs an interconnect it does not have. Use the chart to build a shortlist, then the GPU model pages to find providers with capacity.
Consumer cards are off the chart by default, and they would otherwise lead it: the RTX 3090 rents at $0.0053 per GB-hour, 2.2x cheaper per gigabyte than the RTX A6000. They carry no ECC, no service guarantee, and often a marketplace host rather than a datacenter. Fine for dev boxes and batch work that tolerates a failed run, and the class filter above the chart adds them.
Frequently asked questions
What is cost per GB-hour?
Cost per GB-hour is a GPU's hourly rental price divided by its VRAM in gigabytes. Across the 26 datacenter and workstation GPUs on this page the median is $0.0284 per GB-hour. It measures what you pay to hold a model of a given size, independent of how fast the card runs it.
Which GPU is cheapest per GB of VRAM?
The Nvidia RTX A6000 at $0.0116 per GB-hour, from $0.56 per hour for 48 GB across 13 providers. That is 59% below the market median of $0.0284.
Which GPU has the cheapest memory bandwidth?
The Nvidia A4000 at $0.47 per TB/s-hour, against a median of $1.00. Its absolute bandwidth is 0.45 TB/s, so read the figure as value for money rather than as speed: a cheap card with modest bandwidth can lead this ranking.
Are consumer GPUs cheaper to rent than datacenter GPUs?
Per gigabyte, yes: the Nvidia RTX 3090 rents at $0.0053 per GB-hour, 2.2x cheaper than the RTX A6000, the cheapest card in the default view. They forfeit ECC, NVLink, MIG and service guarantees, so a gap either way measures a different product rather than a better deal. They are not shown on the chart by default; the class filter above it adds them.
Why do the two rankings disagree?
Because the market prices memory size and memory speed separately. The B200 ranks 20 of 26 per gigabyte and 8 of 26 per unit of bandwidth. Which ranking applies to you depends on whether your workload runs out of room or runs out of speed first.
Methodology
Provider-weighted medians for the week of August 17, 2026: each provider's own median first, then the median across providers, for every GPU listed by at least 3 providers that week. Updated weekly. Findings and rankings are organic, not paid placements.
Each axis carries one published spec, never a blended score, because a composite's ranking would depend on how the two specs were weighted rather than on the market.
A median is not the lowest price you can pay. The cheapest GPU cloud guide tracks the single lowest listing per GPU instead, so the same card shows a lower price and a higher provider count there. For historical prices, see the GPU Price Index.