Cloud GPU: on-demand, reserved and spot, and why neoclouds change the board (with 2026 prices)
Contents
Notation: amounts in N USD or N €, decimal point. The dollar sign is not used (on this site it is the formula delimiter). All cloud prices change without notice; the date of the source is given next to each figure.
TL;DR
An H100 SXM5 on-demand costs between 2.49 USD/GPU-hour (Lambda, a neocloud) and 12.29 USD/GPU-hour (Azure ND H100 v5) depending on the provider, a spread of 5× between the neocloud floor and the hyperscaler ceiling. The reserved mode (a 1-3 year commitment) lowers the price by 20-40 % and is the number comparable to amortising your own iron; the spot/preemptible mode takes another 40-65 % off, but with a risk of interruption on less than 2 minutes’ notice. For the on-prem vs cloud analysis of this series (article sovereign on-premise vs hyperscalers), neocloud reserved is the correct reference: at 1.49-2.10 USD/GPU-hour with a 3-year commitment, neoclouds compete head-on with the TCO of your own iron at medium utilisation.
The three cloud GPU purchasing models
On-demand
Pay per use with no commitment, billed by the hour or by the minute. Maximum flexibility; maximum price. Applies to: development, experimentation, occasional capacity peaks and any workload whose lifetime is unpredictable. The absence of commitment has a cost: the on-demand premium over reserved ranges from 25 % to 60 % depending on the provider.
Reserved / committed
A 1 or 3 year contract that guarantees capacity in exchange for a discount on the on-demand rate. At hyperscalers it goes by the name Reserved Instance (AWS), Committed Use Discount (GCP) or Azure Reservation. At neoclouds it is negotiated directly and can include the choice of node, network (Ethernet or InfiniBand) and region. Typical discounts:
| Term | Discount on on-demand (market reference) |
|---|---|
| 1 year | 20-37 % |
| 3 years | 30-45 % |
Reserved is the model comparable to on-prem: in both cases a fixed cost is incurred up front (amortised capex on-prem; committed payment in the cloud). The formula for the effective cost per GPU-hour is the same in both cases:
$$\text{cost/GPU-hour}_{reserved} = \frac{\text{committed price (USD/GPU-h)} \times 8760}{\text{contracted hours}}$$Unlike on-prem, cloud reserved includes neither hardware capex nor operational opex, but neither does it include the benefits of running your own iron at high load (see GPU utilisation as a FinOps lever).
Spot / preemptible
Idle provider capacity offered at a 40-65 % discount on on-demand, on the condition that it can be reclaimed with 30 seconds to 2 minutes of notice (AWS Spot, GCP Spot VMs). The discount is real and reproducible; so is the interruption risk.
Workloads suitable for spot:
| Workload | Requires checkpointing | Suitable for spot |
|---|---|---|
| LLM pre-training (checkpoint-resume every 15-30 min) | yes | yes |
| LoRA/QLoRA fine-tuning with checkpoints | yes | yes |
| Offline batch inference (embeddings, evaluations) | no (idempotent) | yes |
| Hyperparameter search | no (repeatable) | yes |
| Production serving with a latency SLO | not applicable | no |
| Production serving with an availability SLO | not applicable | no |
A fine-tune of a 7B model on 4× H100 over 40 hours on spot achieved real savings of 81 % against the equivalent on-demand (Spheron Blog, May 2026). The typical interruption rate on high-reliability H100 instances is 2-8 % per day, which gives a 92-98 % probability of completing a 24-hour shift without a cut (Thunder Compute, Jun. 2026).
The neoclouds: GPU-specialised clouds
Definition and differentiators
Neoclouds (or GPU-specialised clouds) are infrastructure providers focused exclusively on GPU compute for AI workloads, without the diversity of services of the hyperscalers. SemiAnalysis coined the term in 2024 and identifies CoreWeave, Nebius, Lambda and Crusoe at the top tier (SemiAnalysis, 2024).
Structural differences with respect to the hyperscalers:
| Dimension | Neoclouds (CoreWeave, Lambda, Nebius…) | Hyperscalers (AWS, GCP, Azure) |
|---|---|---|
| H100 on-demand price | 2.49-6.16 USD/GPU-hour | 6.88-12.29 USD/GPU-hour |
| Interconnect network | InfiniBand NDR 400 Gbps (CoreWeave, Voltage Park), Ethernet 200+ Gbps (Lambda, Nebius) | Proprietary Ethernet (EFA, GVNIC) or InfiniBand (only p5/H100 on AWS) |
| Access to new hardware | NVIDIA priority access (CoreWeave is an NVIDIA shareholder); H200/B200 catalogue available earlier | Slower catalogue; B200 in reserved or limited capacity in 2026 |
| Contracts | 1-36 months, negotiated directly | Standard RI (1-3 years), no node negotiation |
| Additional services | Managed Kubernetes, storage, bare-metal | Complete ecosystem (IAM, VPC, databases, ML services…) |
| Egress | Zero-egress or flat fee (Lambda, Voltage Park, Nebius) | 0.08-0.12 USD/GB (AWS, GCP, Azure) |
| Sovereignty | Varies: Nebius (EU), Crusoe (US, clean), CoreWeave (US/EU) | US CLOUD Act; no guarantee of EU jurisdiction |
The neocloud market is estimated at 20,000 M USD of revenue in 2026 and is growing towards 180,000 M USD in 2030 (Synergy Research Group, 2026). CoreWeave passed 5,000 M USD of ARR faster than any other cloud platform. Neoclouds price their H100s 60-85 % below AWS and Azure for the same silicon (Signisys, 2026).
Profiles of the main neoclouds
| Provider | Main hardware | Network | Jurisdiction | Relevant note |
|---|---|---|---|---|
| CoreWeave | H100, H200, B200, GB200 | InfiniBand NDR (Quantum-2 SHARP) | US / EU | NVIDIA shareholder; priority access to hardware; largest neocloud by fleet |
| Lambda Labs | H100, A100, B200 | Ethernet | US | Zero-egress; no spot; per-minute billing |
| Nebius | H100, H200, B200, B300 | InfiniBand | EU (Amsterdam) | Spin-off of Yandex Cloud; zero-egress; object egress 0.015 USD/GiB; public on-demand price |
| Crusoe | H100, A100, MI300X | Ethernet | US | 100 % renewable/flare gas energy; AMD MI300X at 3.45 USD/hr |
| Voltage Park | H100 | Ethernet (1.99 USD) / InfiniBand 3200 Gbps (2.49 USD) | US | Merged with Lightning AI in Jan. 2026; no minimums; non-profit Navigation Fund |
| Together AI | H100, A100 | Ethernet | US | Inference platform + raw compute |
| GMI Cloud | H100, H200 | Ethernet | Asia/US | H200 on-demand from 2.60 USD/hr |
| RunPod | H100, H200, B200, A100 | Ethernet / Secure Cloud | Global | Community + Secure Cloud with SLA; spot available |
Note: the “zero-egress”, “renewable” and priority hardware access claims are marketing from each provider and have not been verified by independent audits. They are reproduced with that caveat.
2026 price table: H100 SXM by provider and mode
Source and date given for each figure. Prices change without notice.
H100 SXM5 (80 GB HBM3) — USD per GPU-hour
| Provider | On-demand | 1-year reserved | 3-year reserved | Spot/Preemptible | Source and date |
|---|---|---|---|---|---|
| CoreWeave | ~6.16 | ~2.10-3.00 (negotiated) | ~1.49-2.10 (negotiated) | not available | Thunder Compute, Jun. 2026 |
| Lambda Labs | 2.49-3.44 | ~1.89 (37 % disc.) | ~1.84 (PCIe 3-yr) | not available | Spheron Blog, May 2026 |
| Nebius | 3.85 | discount up to 35 % (contact) | discount up to 35 % (contact) | 2.15 (preemptible) | nebius.com/prices, Jun. 2026 |
| Voltage Park | 1.99 (Ethernet) / 2.49 (InfiniBand) | contact (≥6 months) | contact | not available | voltagepark.com/pricing, Jun. 2026 |
| RunPod (Secure) | 3.29 | not available | not available | available (~59 % disc.) | Spheron Blog, May 2026 |
| Spheron | 2.50 | volume discount (contact) | volume discount (contact) | 1.03 | Spheron pricing, May 2026 |
| AWS (p5.48xlarge) | 6.88 | ~2.97 (3-yr, ~23.78 USD/h node ÷ 8) | ~2.97 | ~1.14 (p5 spot, 9.10 USD/h node) | Vantage, Jun. 2026 |
| GCP (A3 High) | 10.98 | ~8.78 (1-yr CUD) | n/a | ~3.69 | Spheron Blog, May 2026 |
| Azure (ND H100 v5) | 12.29 | ~6.15-8.60 (1-3 yr reservation) | ~6.15-8.60 | ~2.27 (node spot ÷ 8) | Vantage Azure, Jun. 2026 |
All amounts in USD/GPU-hour, rounded to 2 decimals. The “3-year reserved” column is the number comparable to the amortisation cost of your own hardware.
AWS p5.48xlarge: an 8× H100 node; on-demand 55.04 USD/h → 6.88 USD/GPU-h; 3-yr reserved 23.78 USD/h → 2.97 USD/GPU-h; spot 9.10-25.35 USD/h (varies by region and availability). GCP a3-highgpu-8g: 8× H100 SXM5; on-demand 87.84 USD/h → 10.98 USD/GPU-h; spot ~3.69 USD/GPU-h (us-central1, May 2026). Azure ND96isr H100 v5: 8× H100; on-demand 98.32 USD/h → 12.29 USD/GPU-h; node spot 18.17 USD/h → ~2.27 USD/GPU-h (source: Vantage, Jun. 2026, us-east-1).
2026 price table: H200 and B200
H200 SXM (141 GB HBM3e) — USD per GPU-hour
| Provider | On-demand | Spot/Preemptible | Source and date |
|---|---|---|---|
| Nebius | 4.50 | 2.45 (preemptible) | nebius.com/prices, Jun. 2026 |
| GMI Cloud | 2.60 | not available | GMI Cloud, May 2026 |
| RunPod | 4.39 | not available | Spheron Blog, May 2026 |
| Spheron | 4.54 | not available | Spheron pricing, May 2026 |
| CoreWeave | 6.31 | not available | GMI Cloud, May 2026 |
| AWS (p5e) | ~4.98 | limited | Spheron Blog, May 2026 |
| Azure | ~13.78 | not available | Spheron Blog, May 2026 |
B200 SXM6 (192 GB HBM3e) — USD per GPU-hour
| Provider | On-demand | Spot | Source and date |
|---|---|---|---|
| Lambda Labs | 4.99-5.29 | not available | Spheron Blog, May 2026 |
| Nebius (HGX B200) | 7.15 | 3.95 (preemptible) | nebius.com/prices, Jun. 2026 |
| RunPod | 5.89 | not available | Spheron Blog, May 2026 |
| Spheron | 6.02 | 2.12 | Spheron pricing, May 2026 |
| AWS (p6-B200) | ~14.24 | ~3.24 | Spheron Blog, May 2026 |
B200 availability is still limited in Q2 2026; many providers offer only access by advance reservation. The prices are market estimates subject to rapid change.
How it fits into the on-prem vs cloud comparison
The cloud cost per GPU-hour is the number that goes head to head with the on-prem TCO computed in sovereign on-premise vs hyperscalers. The relationship between the three purchasing modes and the cost analysis is as follows:
| Cloud mode | What it represents in the comparison | On-prem equivalent |
|---|---|---|
| On-demand | marginal cost of peak capacity; floor of build-vs-buy at 100 % utilisation | — (not comparable: on-prem has no elasticity) |
| 3-year reserved | the committed cost comparable to capex amortised over 3 years | node amortisation + fixed annual opex |
| Spot | the cloud price floor, but with interruption risk | — (not comparable: on-prem is not interrupted) |
The cost per token formula applies the same to cloud as to on-prem:
$$\text{cost/1M tokens}_{cloud} = \frac{\text{GPU-hour price (USD)} \times 10^6}{\text{throughput (tok/s)} \times 3600}$$With an H100 SXM5 at 2,800 tok/s (typical throughput for a 70B model in FP8):
| Mode | Reference price | Cost/1M tokens (70B, 2800 tok/s) |
|---|---|---|
| Neocloud on-demand (Lambda) | 2.49 USD/GPU-h | ~0.247 USD (~0.23 €) |
| Neocloud 3-yr reserved (CoreWeave) | ~1.49 USD/GPU-h | ~0.148 USD (~0.14 €) |
| Neocloud spot (Spheron/Nebius) | ~1.03-2.15 USD/GPU-h | ~0.102-0.213 USD |
| AWS on-demand (p5) | 6.88 USD/GPU-h | ~0.682 USD (~0.63 €) |
| AWS 3-yr reserved (p5) | ~2.97 USD/GPU-h | ~0.295 USD (~0.27 €) |
| GCP spot (A3) | 3.69 USD/GPU-h | ~0.366 USD (~0.34 €) |
| On-prem amortised, high util. (80 %) | ~2.83 USD/GPU-h all-in | ~0.281 USD (~0.26 €) |
Reference exchange rate: 1 USD = 0.924 €. The rate fluctuates; comparative calculations must be done in the same currency.
The key reading: at high utilisation, on-prem TCO (~2.83 USD/GPU-hour all-in according to Spheron, 2026) competes with the 3-year reserved of the neocloud (~1.49-2.97 USD/GPU-hour). On-prem wins at high utilisation when the all-in cost drops below 2.5 USD; neocloud reserved wins when utilisation is medium or low, because it does not pay for idle capex. Spot lowers the cloud floor but is not comparable: interruption makes it incompatible with continuous serving, which is the case where on-prem competes.
Network factors: InfiniBand vs Ethernet
For distributed training and large-scale inference (tensor parallelism across nodes), the bandwidth of the interconnect network between GPUs is critical. Neoclouds differ in what they offer:
| Provider | GPU-GPU network | Bandwidth | Applies to |
|---|---|---|---|
| CoreWeave | InfiniBand NDR (Quantum-2, SHARP) | 400 Gbps/GPU | multi-node HGX clusters |
| Voltage Park | InfiniBand Quantum-2 | 3200 Gbps (8-GPU node) | the 2.49 USD/hr tier |
| Voltage Park | Ethernet | 200 Gbps | the 1.99 USD/hr tier |
| Lambda Labs | Ethernet | 200 Gbps | all nodes |
| Nebius | InfiniBand | 400 Gbps/GPU | HGX H100/H200/B200 |
| AWS p5 | EFA (Elastic Fabric Adapter) | 3200 Gbps (node) | EC2 clusters |
| GCP A3 | GVNIC + RoCE | 200 Gbps/GPU | A3 High |
| Azure ND H100 v5 | InfiniBand NDR | 400 Gbps/GPU | ND96isr H100 v5 |
InfiniBand NDR with SHARP (in-network collective reduction) cuts the latency of allreduce operations in distributed training. For inference serving with tensor parallelism inside a node (NVLink), the inter-node network is less critical; for multi-node training or disaggregated prefill-decode (the three axes of the series), the network between nodes matters.
Neoclouds and sovereignty: what changes
Unlike the US hyperscalers subject to the US CLOUD Act, some neoclouds offer options with jurisdiction outside the US:
| Provider | EU region | US CLOUD Act applies |
|---|---|---|
| Nebius | Amsterdam (NL), Frankfurt (DE) | No (Dutch legal entity) |
| CoreWeave | Frankfurt (DE), London (GB) | Yes (US company) |
| Lambda Labs | US only | Yes |
| Voltage Park | US only | Yes |
| Scaleway | Paris (FR), Amsterdam (NL) | No (French company) |
| US hyperscalers | EU regions available | Yes (CLOUD Act applies) |
For data subject to GDPR, only providers with an EU/EFTA legal entity guarantee the absence of the CLOUD Act. The full discussion of the sovereignty axis is in sovereign on-premise vs hyperscalers. For capacity planning with this data, see capacity planning for on-premise inference.
Decision table: purchasing mode by cost/risk/sovereignty axis
| Mode | Cost | Operational risk | Sovereignty (EU) | When to use |
|---|---|---|---|---|
| Neocloud on-demand | medium (2-4 USD/GPU-h) | low (no interruption) | depends on the provider | development, unpredictable peaks, serving with an SLO |
| Neocloud 1-yr reserved | low-medium (1.5-3 USD/GPU-h) | low (no interruption, guaranteed capacity) | depends on the provider | stable serving, scheduled training; validated workload |
| Neocloud 3-yr reserved | low (1.5-2.1 USD/GPU-h) | low, financial commitment | depends on the provider | comparable to amortised on-prem; very stable base load |
| Neocloud spot | minimum (1-2.5 USD/GPU-h) | high (interruption <2 min) | depends on the provider | batch, fine-tuning with checkpoints, hyperparameter search |
| Hyperscaler on-demand | high (6.9-12.3 USD/GPU-h) | low | NO (CLOUD Act) | integration with AWS/GCP/Azure services; no alternative |
| Hyperscaler reserved | medium-high (3-9 USD/GPU-h) | low | NO (CLOUD Act) | lock-in to the hyperscaler ecosystem is justified |
| Hyperscaler spot | medium (1.1-3.7 USD/GPU-h) | high (interruption) | NO (CLOUD Act) | batch with no sovereignty requirement; more availability than neocloud spot |
| Amortised on-prem (>70 % util.) | low (~2.3-2.8 USD/GPU-h all-in) | low (no interruption) | total (your own hardware) | stable, high base load; maximum sovereignty |
On-prem competes with neocloud 3-yr reserved when utilisation exceeds 65-70 % (see GPU utilisation as FinOps).
Computing the spot saving: an example with checkpointing
Reference parameters to validate whether spot is worth it:
$$\text{spot saving} = (\text{OD price} - \text{spot price}) \times \text{total GPU-hours} - \text{retry cost}$$ $$\text{retry cost} = \text{interruption rate} \times \text{lost work (GPU-hours)} \times \text{spot price}$$With checkpointing every 30 minutes, work lost per interruption is ≤ 0.5 GPU-hours per event. At an interruption rate of 4 % per day on one H100 GPU:
| Item | Reference value |
|---|---|
| On-demand price (neocloud) | 2.50 USD/GPU-h |
| Spot price | 1.03 USD/GPU-h |
| Gross saving per GPU-hour | 1.47 USD |
| Expected interruptions in 40 h (4 %/day) | ~0.67 events |
| Re-run work per event (0.5 h × 1.03 USD) | ~0.34 USD |
| Net saving on a 40 h × 4 GPU job | ~234 USD (against 400 USD on-demand) |
The 58 % saving only materialises if the job implements checkpoint-resume correctly. Without checkpoints, a single interruption forces a restart from scratch: the saving turns into an overcost from duplicated work.
Integration into the series TCO model
The cloud €/GPU-hour is the number that feeds the FinOps analysis of the series. The chain:
- Cost of cloud iron (this price table) → equivalent to amortised capex + opex of on-prem.
- Throughput (measured with genai-perf, track B of the series) → the denominator.
- Cost per token (cost per token and per request) → the final comparative metric.
- Utilisation (GPU utilisation as FinOps) → in on-demand cloud, utilisation does not penalise cost (you pay for what you use); in reserved and in on-prem it does penalise, because the cost is fixed.
The correct comparison is neocloud 3-yr reserved vs on-prem amortised over 3 years, with the same assumptions on throughput, egress and utilisation. Spot is the cloud price floor but not the comparable floor: its interruptible nature excludes it from any comparison with an on-prem cluster that serves continuous traffic.
Sources
- Spheron Blog · GPU Cloud Pricing 2026: H100 from 1,03 USD/hr, B200 from 2,12 USD/hr (15+ providers), may. 2026 — https://www.spheron.network/blog/gpu-cloud-pricing-comparison-2026/
- Vantage · p5.48xlarge pricing and specs (AWS), actualizado jun. 2026 — https://instances.vantage.sh/aws/ec2/p5.48xlarge
- Nebius · NVIDIA GPU Pricing (página oficial, jun. 2026) — https://nebius.com/prices
- Voltage Park · H100 GPU Pricing (página oficial, jun. 2026) — https://www.voltagepark.com/pricing
- Thunder Compute · CoreWeave GPU Pricing Review (jun. 2026) — https://www.thundercompute.com/blog/coreweave-gpu-pricing-review
- Thunder Compute · NVIDIA H100 Pricing (Jun 2026) — https://www.thundercompute.com/blog/nvidia-h100-pricing
- Spheron Blog · Lambda Cloud H100 Pricing 2026 — https://www.spheron.network/blog/lambda-cloud-h100-pricing-2026/
- Spheron Blog · AWS H100 Pricing 2026: P5 Instance Cost — https://www.spheron.network/blog/aws-h100-pricing-2026/
- Spheron Blog · Google Cloud A3 H100 Pricing 2026 — https://www.spheron.network/blog/google-cloud-a3-h100-pricing/
- Spheron Blog · Azure H100 Pricing 2026: ND H100 v5 — https://www.spheron.network/blog/azure-h100-pricing/
- GMI Cloud · H200 GPU Provider Pricing 2026 (CoreWeave, Lambda, Nebius, GMI) — https://www.gmicloud.ai/en/blog/h200-gpu-provider-pricing
- Spheron Blog · GPU Spot Instance Arbitrage 2026 — https://www.spheron.network/blog/gpu-spot-instance-arbitrage-2026/
- Thunder Compute · Cloud GPU Spot Instance Availability and Interruption Rates — https://www.thundercompute.com/blog/cloud-gpu-spot-instance-availability
- Synergy Research Group · Neocloud Market Forecast to Approach 400B USD by 2031 — https://www.srgresearch.com/articles/neocloud-market-forecast-to-approach-400b-by-2031-driven-by-surging-ai-infrastructure-demand
- Signisys · The Neocloud Revolution: How 20 Billion USD in GPU Providers Reshapes Cloud Market — https://www.signisys.com/blog/the-neocloud-revolution-how-20-billion-in-gpu-focused-providers-are-reshaping-the-cloud-market/
- SemiAnalysis · The Great GPU Shortage: Rental Capacity (neocloud taxonomy original) — https://newsletter.semianalysis.com/p/the-great-gpu-shortage-rental-capacity
- Spheron Blog · LLM Inference On-Premise vs GPU Cloud: 2026 Cost and Break-Even — https://www.spheron.network/blog/llm-inference-on-premise-vs-cloud/
- IntuitionLabs · H100 Rental Prices Compared 1,49-6,98 USD/hr Across 15+ Cloud Providers (2026) — https://intuitionlabs.ai/articles/h100-rental-prices-cloud-comparison