Intel's data center AI accelerator with on-chip Ethernet for cost-efficient scale-out.
Weekly median price per GPU per hour
By provider shows one card per company, placed where its best offer ranked. By configuration lists every offer.
What you can rent comes first: in stock, then waitlist, then not reported, then out of stock. Priced offers rank ahead of quote-only, and on-demand ahead of other billing types.
Within a group, five factors set the order:
We match the provider, its country, GPU model, form factor, billing type, availability and instance name. Matching is partial and case-insensitive. Everyday words work too, so "interruptible" finds spot and "sold out" finds out of stock.
Heads up: A provider's own page may quote a different figure, for example on tax, region or a promotion. A monthly-only plan shows a derived hourly rate. Verify before provisioning. More on how we price.
One Gaudi2 has 96 GB of VRAM. In practice, that's enough memory for roughly 146B parameters at 4-bit or 38B at 16-bit, assuming a 32K context. Below are some open-weight LLMs, with the estimated memory and GPUs each one needs.
| Model | Memory (INT4 / FP4) | Gaudi2s needed | Cost /hr | Cost /mo |
|---|---|---|---|---|
Qwen3.8-27B
27B
|
18 GB
|
1
|
$1.21
|
$871
|
Gemma 4 31B
30.7B
|
22 GB
|
1
|
$1.21
|
$871
|
GPT-OSS-120B
117B · 5.1B active
|
67 GB
|
1
|
$1.21
|
$871
|
GLM-5.3-Flash
320B · 18B active
|
178 GB
|
8
|
$9.68
|
$6,970
|
MiniMax-M3
428B · 23B active
|
239 GB
|
8
|
$9.68
|
$6,970
|
DeepSeek V4.1 Flash
552B · 16B active
|
306 GB
|
8
|
$9.68
|
$6,970
|
Kimi K3
2.8T · 104B active
|
1,544 GB
|
18
|
–
|
–
|
Estimates based on the median on-demand rate. Memory is weights plus FP8 KV cache at 32K context per request (FP16/BF16 in the 16-bit column). GPU counts assume 90% of advertised VRAM is usable. No guarantee of runtime support, usable performance, or that a matching quantized build exists. How we estimate costs.
Intel Gaudi2 · Per GPU
| Compute · dense | |
|---|---|
| FP8 | 865 TFLOPS |
| FP16 / BF16 | 432 TFLOPS |
| Precision support | FP8FP16BF16TF32FP32INT8 |
| Memory | |
|---|---|
| Capacity | 96 GB HBM2e |
| Bandwidth | 2,450 GB/s |
| Silicon | |
|---|---|
| Architecture | Gaudi (2nd Gen) |
| Process | TSMC 7nm |
| Fabric and host | |
|---|---|
| GPU interconnect | RoCE 600 GB/s |
| Host interface | PCIe 4.0 x16 |
| Power | |
|---|---|
| Board power | 600 W |
Source: official Intel Gaudi2 datasheet.
As of September 12, 2026, the median on-demand price is $1.21 per GPU per hour across 3 providers.
| Billing type | Configs | Median /GPU/hr | Cheapest |
|---|---|---|---|
On-demand | 6 | $1.21 | $1.03 |
Reserved | 12 | $0.58 (12 mo) |
No median for reserved: we only show this when at least 3 providers list the GPU on that billing type.
The lowest listed on-demand price for the Gaudi2 is $1.03 per GPU per hour from Cyfuture AI, though we haven't verified current stock. Cheapest verified in stock: on-demand $1.21 from Runcrate.
At 720 hours per month, one Gaudi2 costs an estimated $871 at the median on-demand price. Cheapest verified in stock: $871 per month on-demand.
GetDeploying currently tracks Gaudi2 configs from 3 providers. Runcrate and Sesterce have verified in-stock on-demand configs. See the full price comparison above for every provider and config.
One Gaudi2 runs models up to roughly 146B parameters at 4-bit quantization or 38B at 16-bit, assuming a 32K context. Larger models run across multiple GPUs: the model table above shows the estimated memory and GPU count for popular open-weight LLMs.
Comes with 96GB HBM2e and 24x 100GbE RoCEv2 ports integrated on-chip, so clusters scale out over standard Ethernet without separate NICs. FP8 support and competitive pricing make it a cost-efficient option for training and inference on the Intel Gaudi software stack.
Last updated
