Cloud GPU: on-demand, reserved and spot, and why neoclouds change the board (with 2026 prices)

Tue, 16 Jun 2026 05:00:00 +0200

Notation: amounts in N USD or N €, decimal point. The dollar sign is not used (on this site it is the formula delimiter). All cloud prices change without notice; the date of the source is given next to each figure.

TL;DR

An H100 SXM5 on-demand costs between 2.49 USD/GPU-hour (Lambda, a neocloud) and 12.29 USD/GPU-hour (Azure ND H100 v5) depending on the provider, a spread of 5× between the neocloud floor and the hyperscaler ceiling. The reserved mode (a 1-3 year commitment) lowers the price by 20-40 % and is the number comparable to amortising your own iron; the spot/preemptible mode takes another 40-65 % off, but with a risk of interruption on less than 2 minutes’ notice. For the on-prem vs cloud analysis of this series (article sovereign on-premise vs hyperscalers), neocloud reserved is the correct reference: at 1.49-2.10 USD/GPU-hour with a 3-year commitment, neoclouds compete head-on with the TCO of your own iron at medium utilisation.

The three cloud GPU purchasing models

On-demand

Pay per use with no commitment, billed by the hour or by the minute. Maximum flexibility; maximum price. Applies to: development, experimentation, occasional capacity peaks and any workload whose lifetime is unpredictable. The absence of commitment has a cost: the on-demand premium over reserved ranges from 25 % to 60 % depending on the provider.

Reserved / committed

A 1 or 3 year contract that guarantees capacity in exchange for a discount on the on-demand rate. At hyperscalers it goes by the name Reserved Instance (AWS), Committed Use Discount (GCP) or Azure Reservation. At neoclouds it is negotiated directly and can include the choice of node, network (Ethernet or InfiniBand) and region. Typical discounts:

Term	Discount on on-demand (market reference)
1 year	20-37 %
3 years	30-45 %

Reserved is the model comparable to on-prem: in both cases a fixed cost is incurred up front (amortised capex on-prem; committed payment in the cloud). The formula for the effective cost per GPU-hour is the same in both cases:

$$\text{cost/GPU-hour}_{reserved} = \frac{\text{committed price (USD/GPU-h)} \times 8760}{\text{contracted hours}}$$

Unlike on-prem, cloud reserved includes neither hardware capex nor operational opex, but neither does it include the benefits of running your own iron at high load (see GPU utilisation as a FinOps lever).

Spot / preemptible

Idle provider capacity offered at a 40-65 % discount on on-demand, on the condition that it can be reclaimed with 30 seconds to 2 minutes of notice (AWS Spot, GCP Spot VMs). The discount is real and reproducible; so is the interruption risk.

Workloads suitable for spot:

Workload	Requires checkpointing	Suitable for spot
LLM pre-training (checkpoint-resume every 15-30 min)	yes	yes
LoRA/QLoRA fine-tuning with checkpoints	yes	yes
Offline batch inference (embeddings, evaluations)	no (idempotent)	yes
Hyperparameter search	no (repeatable)	yes
Production serving with a latency SLO	not applicable	no
Production serving with an availability SLO	not applicable	no

A fine-tune of a 7B model on 4× H100 over 40 hours on spot achieved real savings of 81 % against the equivalent on-demand (Spheron Blog, May 2026). The typical interruption rate on high-reliability H100 instances is 2-8 % per day, which gives a 92-98 % probability of completing a 24-hour shift without a cut (Thunder Compute, Jun. 2026).

The neoclouds: GPU-specialised clouds

Definition and differentiators

Neoclouds (or GPU-specialised clouds) are infrastructure providers focused exclusively on GPU compute for AI workloads, without the diversity of services of the hyperscalers. SemiAnalysis coined the term in 2024 and identifies CoreWeave, Nebius, Lambda and Crusoe at the top tier (SemiAnalysis, 2024).

Structural differences with respect to the hyperscalers:

Dimension	Neoclouds (CoreWeave, Lambda, Nebius…)	Hyperscalers (AWS, GCP, Azure)
H100 on-demand price	2.49-6.16 USD/GPU-hour	6.88-12.29 USD/GPU-hour
Interconnect network	InfiniBand NDR 400 Gbps (CoreWeave, Voltage Park), Ethernet 200+ Gbps (Lambda, Nebius)	Proprietary Ethernet (EFA, GVNIC) or InfiniBand (only p5/H100 on AWS)
Access to new hardware	NVIDIA priority access (CoreWeave is an NVIDIA shareholder); H200/B200 catalogue available earlier	Slower catalogue; B200 in reserved or limited capacity in 2026
Contracts	1-36 months, negotiated directly	Standard RI (1-3 years), no node negotiation
Additional services	Managed Kubernetes, storage, bare-metal	Complete ecosystem (IAM, VPC, databases, ML services…)
Egress	Zero-egress or flat fee (Lambda, Voltage Park, Nebius)	0.08-0.12 USD/GB (AWS, GCP, Azure)
Sovereignty	Varies: Nebius (EU), Crusoe (US, clean), CoreWeave (US/EU)	US CLOUD Act; no guarantee of EU jurisdiction

The neocloud market is estimated at 20,000 M USD of revenue in 2026 and is growing towards 180,000 M USD in 2030 (Synergy Research Group, 2026). CoreWeave passed 5,000 M USD of ARR faster than any other cloud platform. Neoclouds price their H100s 60-85 % below AWS and Azure for the same silicon (Signisys, 2026).

Profiles of the main neoclouds

Provider	Main hardware	Network	Jurisdiction	Relevant note
CoreWeave	H100, H200, B200, GB200	InfiniBand NDR (Quantum-2 SHARP)	US / EU	NVIDIA shareholder; priority access to hardware; largest neocloud by fleet
Lambda Labs	H100, A100, B200	Ethernet	US	Zero-egress; no spot; per-minute billing
Nebius	H100, H200, B200, B300	InfiniBand	EU (Amsterdam)	Spin-off of Yandex Cloud; zero-egress; object egress 0.015 USD/GiB; public on-demand price
Crusoe	H100, A100, MI300X	Ethernet	US	100 % renewable/flare gas energy; AMD MI300X at 3.45 USD/hr
Voltage Park	H100	Ethernet (1.99 USD) / InfiniBand 3200 Gbps (2.49 USD)	US	Merged with Lightning AI in Jan. 2026; no minimums; non-profit Navigation Fund
Together AI	H100, A100	Ethernet	US	Inference platform + raw compute
GMI Cloud	H100, H200	Ethernet	Asia/US	H200 on-demand from 2.60 USD/hr
RunPod	H100, H200, B200, A100	Ethernet / Secure Cloud	Global	Community + Secure Cloud with SLA; spot available

Note: the “zero-egress”, “renewable” and priority hardware access claims are marketing from each provider and have not been verified by independent audits. They are reproduced with that caveat.

2026 price table: H100 SXM by provider and mode

Source and date given for each figure. Prices change without notice.

H100 SXM5 (80 GB HBM3) — USD per GPU-hour

Provider	On-demand	1-year reserved	3-year reserved	Spot/Preemptible	Source and date
CoreWeave	~6.16	~2.10-3.00 (negotiated)	~1.49-2.10 (negotiated)	not available	Thunder Compute, Jun. 2026
Lambda Labs	2.49-3.44	~1.89 (37 % disc.)	~1.84 (PCIe 3-yr)	not available	Spheron Blog, May 2026
Nebius	3.85	discount up to 35 % (contact)	discount up to 35 % (contact)	2.15 (preemptible)	nebius.com/prices, Jun. 2026
Voltage Park	1.99 (Ethernet) / 2.49 (InfiniBand)	contact (≥6 months)	contact	not available	voltagepark.com/pricing, Jun. 2026
RunPod (Secure)	3.29	not available	not available	available (~59 % disc.)	Spheron Blog, May 2026
Spheron	2.50	volume discount (contact)	volume discount (contact)	1.03	Spheron pricing, May 2026
AWS (p5.48xlarge)	6.88	~2.97 (3-yr, ~23.78 USD/h node ÷ 8)	~2.97	~1.14 (p5 spot, 9.10 USD/h node)	Vantage, Jun. 2026
GCP (A3 High)	10.98	~8.78 (1-yr CUD)	n/a	~3.69	Spheron Blog, May 2026
Azure (ND H100 v5)	12.29	~6.15-8.60 (1-3 yr reservation)	~6.15-8.60	~2.27 (node spot ÷ 8)	Vantage Azure, Jun. 2026

All amounts in USD/GPU-hour, rounded to 2 decimals. The “3-year reserved” column is the number comparable to the amortisation cost of your own hardware.

AWS p5.48xlarge: an 8× H100 node; on-demand 55.04 USD/h → 6.88 USD/GPU-h; 3-yr reserved 23.78 USD/h → 2.97 USD/GPU-h; spot 9.10-25.35 USD/h (varies by region and availability). GCP a3-highgpu-8g: 8× H100 SXM5; on-demand 87.84 USD/h → 10.98 USD/GPU-h; spot ~3.69 USD/GPU-h (us-central1, May 2026). Azure ND96isr H100 v5: 8× H100; on-demand 98.32 USD/h → 12.29 USD/GPU-h; node spot 18.17 USD/h → ~2.27 USD/GPU-h (source: Vantage, Jun. 2026, us-east-1).

2026 price table: H200 and B200

H200 SXM (141 GB HBM3e) — USD per GPU-hour

Provider	On-demand	Spot/Preemptible	Source and date
Nebius	4.50	2.45 (preemptible)	nebius.com/prices, Jun. 2026
GMI Cloud	2.60	not available	GMI Cloud, May 2026
RunPod	4.39	not available	Spheron Blog, May 2026
Spheron	4.54	not available	Spheron pricing, May 2026
CoreWeave	6.31	not available	GMI Cloud, May 2026
AWS (p5e)	~4.98	limited	Spheron Blog, May 2026
Azure	~13.78	not available	Spheron Blog, May 2026

B200 SXM6 (192 GB HBM3e) — USD per GPU-hour

Provider	On-demand	Spot	Source and date
Lambda Labs	4.99-5.29	not available	Spheron Blog, May 2026
Nebius (HGX B200)	7.15	3.95 (preemptible)	nebius.com/prices, Jun. 2026
RunPod	5.89	not available	Spheron Blog, May 2026
Spheron	6.02	2.12	Spheron pricing, May 2026
AWS (p6-B200)	~14.24	~3.24	Spheron Blog, May 2026

B200 availability is still limited in Q2 2026; many providers offer only access by advance reservation. The prices are market estimates subject to rapid change.

How it fits into the on-prem vs cloud comparison

The cloud cost per GPU-hour is the number that goes head to head with the on-prem TCO computed in sovereign on-premise vs hyperscalers. The relationship between the three purchasing modes and the cost analysis is as follows:

Cloud mode	What it represents in the comparison	On-prem equivalent
On-demand	marginal cost of peak capacity; floor of build-vs-buy at 100 % utilisation	— (not comparable: on-prem has no elasticity)
3-year reserved	the committed cost comparable to capex amortised over 3 years	node amortisation + fixed annual opex
Spot	the cloud price floor, but with interruption risk	— (not comparable: on-prem is not interrupted)

The cost per token formula applies the same to cloud as to on-prem:

$$\text{cost/1M tokens}_{cloud} = \frac{\text{GPU-hour price (USD)} \times 10^6}{\text{throughput (tok/s)} \times 3600}$$

With an H100 SXM5 at 2,800 tok/s (typical throughput for a 70B model in FP8):

Mode	Reference price	Cost/1M tokens (70B, 2800 tok/s)
Neocloud on-demand (Lambda)	2.49 USD/GPU-h	~0.247 USD (~0.23 €)
Neocloud 3-yr reserved (CoreWeave)	~1.49 USD/GPU-h	~0.148 USD (~0.14 €)
Neocloud spot (Spheron/Nebius)	~1.03-2.15 USD/GPU-h	~0.102-0.213 USD
AWS on-demand (p5)	6.88 USD/GPU-h	~0.682 USD (~0.63 €)
AWS 3-yr reserved (p5)	~2.97 USD/GPU-h	~0.295 USD (~0.27 €)
GCP spot (A3)	3.69 USD/GPU-h	~0.366 USD (~0.34 €)
On-prem amortised, high util. (80 %)	~2.83 USD/GPU-h all-in	~0.281 USD (~0.26 €)

Reference exchange rate: 1 USD = 0.924 €. The rate fluctuates; comparative calculations must be done in the same currency.

The key reading: at high utilisation, on-prem TCO (~2.83 USD/GPU-hour all-in according to Spheron, 2026) competes with the 3-year reserved of the neocloud (~1.49-2.97 USD/GPU-hour). On-prem wins at high utilisation when the all-in cost drops below 2.5 USD; neocloud reserved wins when utilisation is medium or low, because it does not pay for idle capex. Spot lowers the cloud floor but is not comparable: interruption makes it incompatible with continuous serving, which is the case where on-prem competes.

Network factors: InfiniBand vs Ethernet

For distributed training and large-scale inference (tensor parallelism across nodes), the bandwidth of the interconnect network between GPUs is critical. Neoclouds differ in what they offer:

Provider	GPU-GPU network	Bandwidth	Applies to
CoreWeave	InfiniBand NDR (Quantum-2, SHARP)	400 Gbps/GPU	multi-node HGX clusters
Voltage Park	InfiniBand Quantum-2	3200 Gbps (8-GPU node)	the 2.49 USD/hr tier
Voltage Park	Ethernet	200 Gbps	the 1.99 USD/hr tier
Lambda Labs	Ethernet	200 Gbps	all nodes
Nebius	InfiniBand	400 Gbps/GPU	HGX H100/H200/B200
AWS p5	EFA (Elastic Fabric Adapter)	3200 Gbps (node)	EC2 clusters
GCP A3	GVNIC + RoCE	200 Gbps/GPU	A3 High
Azure ND H100 v5	InfiniBand NDR	400 Gbps/GPU	ND96isr H100 v5

InfiniBand NDR with SHARP (in-network collective reduction) cuts the latency of allreduce operations in distributed training. For inference serving with tensor parallelism inside a node (NVLink), the inter-node network is less critical; for multi-node training or disaggregated prefill-decode (the three axes of the series), the network between nodes matters.

Neoclouds and sovereignty: what changes

Unlike the US hyperscalers subject to the US CLOUD Act, some neoclouds offer options with jurisdiction outside the US:

Provider	EU region	US CLOUD Act applies
Nebius	Amsterdam (NL), Frankfurt (DE)	No (Dutch legal entity)
CoreWeave	Frankfurt (DE), London (GB)	Yes (US company)
Lambda Labs	US only	Yes
Voltage Park	US only	Yes
Scaleway	Paris (FR), Amsterdam (NL)	No (French company)
US hyperscalers	EU regions available	Yes (CLOUD Act applies)

For data subject to GDPR, only providers with an EU/EFTA legal entity guarantee the absence of the CLOUD Act. The full discussion of the sovereignty axis is in sovereign on-premise vs hyperscalers. For capacity planning with this data, see capacity planning for on-premise inference.

Decision table: purchasing mode by cost/risk/sovereignty axis

Mode	Cost	Operational risk	Sovereignty (EU)	When to use
Neocloud on-demand	medium (2-4 USD/GPU-h)	low (no interruption)	depends on the provider	development, unpredictable peaks, serving with an SLO
Neocloud 1-yr reserved	low-medium (1.5-3 USD/GPU-h)	low (no interruption, guaranteed capacity)	depends on the provider	stable serving, scheduled training; validated workload
Neocloud 3-yr reserved	low (1.5-2.1 USD/GPU-h)	low, financial commitment	depends on the provider	comparable to amortised on-prem; very stable base load
Neocloud spot	minimum (1-2.5 USD/GPU-h)	high (interruption <2 min)	depends on the provider	batch, fine-tuning with checkpoints, hyperparameter search
Hyperscaler on-demand	high (6.9-12.3 USD/GPU-h)	low	NO (CLOUD Act)	integration with AWS/GCP/Azure services; no alternative
Hyperscaler reserved	medium-high (3-9 USD/GPU-h)	low	NO (CLOUD Act)	lock-in to the hyperscaler ecosystem is justified
Hyperscaler spot	medium (1.1-3.7 USD/GPU-h)	high (interruption)	NO (CLOUD Act)	batch with no sovereignty requirement; more availability than neocloud spot
Amortised on-prem (>70 % util.)	low (~2.3-2.8 USD/GPU-h all-in)	low (no interruption)	total (your own hardware)	stable, high base load; maximum sovereignty

On-prem competes with neocloud 3-yr reserved when utilisation exceeds 65-70 % (see GPU utilisation as FinOps).

Computing the spot saving: an example with checkpointing

Reference parameters to validate whether spot is worth it:

$$\text{spot saving} = (\text{OD price} - \text{spot price}) \times \text{total GPU-hours} - \text{retry cost}$$ $$\text{retry cost} = \text{interruption rate} \times \text{lost work (GPU-hours)} \times \text{spot price}$$

With checkpointing every 30 minutes, work lost per interruption is ≤ 0.5 GPU-hours per event. At an interruption rate of 4 % per day on one H100 GPU:

Item	Reference value
On-demand price (neocloud)	2.50 USD/GPU-h
Spot price	1.03 USD/GPU-h
Gross saving per GPU-hour	1.47 USD
Expected interruptions in 40 h (4 %/day)	~0.67 events
Re-run work per event (0.5 h × 1.03 USD)	~0.34 USD
Net saving on a 40 h × 4 GPU job	~234 USD (against 400 USD on-demand)

The 58 % saving only materialises if the job implements checkpoint-resume correctly. Without checkpoints, a single interruption forces a restart from scratch: the saving turns into an overcost from duplicated work.

Integration into the series TCO model

The cloud €/GPU-hour is the number that feeds the FinOps analysis of the series. The chain:

Cost of cloud iron (this price table) → equivalent to amortised capex + opex of on-prem.
Throughput (measured with genai-perf, track B of the series) → the denominator.
Cost per token (cost per token and per request) → the final comparative metric.
Utilisation (GPU utilisation as FinOps) → in on-demand cloud, utilisation does not penalise cost (you pay for what you use); in reserved and in on-prem it does penalise, because the cost is fixed.

The correct comparison is neocloud 3-yr reserved vs on-prem amortised over 3 years, with the same assumptions on throughput, egress and utilisation. Spot is the cloud price floor but not the comparable floor: its interruptible nature excludes it from any comparison with an on-prem cluster that serves continuous traffic.

Sources

Spheron Blog · GPU Cloud Pricing 2026: H100 from 1,03 USD/hr, B200 from 2,12 USD/hr (15+ providers), may. 2026 — https://www.spheron.network/blog/gpu-cloud-pricing-comparison-2026/
Vantage · p5.48xlarge pricing and specs (AWS), actualizado jun. 2026 — https://instances.vantage.sh/aws/ec2/p5.48xlarge
Nebius · NVIDIA GPU Pricing (página oficial, jun. 2026) — https://nebius.com/prices
Voltage Park · H100 GPU Pricing (página oficial, jun. 2026) — https://www.voltagepark.com/pricing
Thunder Compute · CoreWeave GPU Pricing Review (jun. 2026) — https://www.thundercompute.com/blog/coreweave-gpu-pricing-review
Thunder Compute · NVIDIA H100 Pricing (Jun 2026) — https://www.thundercompute.com/blog/nvidia-h100-pricing
Spheron Blog · Lambda Cloud H100 Pricing 2026 — https://www.spheron.network/blog/lambda-cloud-h100-pricing-2026/
Spheron Blog · AWS H100 Pricing 2026: P5 Instance Cost — https://www.spheron.network/blog/aws-h100-pricing-2026/
Spheron Blog · Google Cloud A3 H100 Pricing 2026 — https://www.spheron.network/blog/google-cloud-a3-h100-pricing/
Spheron Blog · Azure H100 Pricing 2026: ND H100 v5 — https://www.spheron.network/blog/azure-h100-pricing/
GMI Cloud · H200 GPU Provider Pricing 2026 (CoreWeave, Lambda, Nebius, GMI) — https://www.gmicloud.ai/en/blog/h200-gpu-provider-pricing
Spheron Blog · GPU Spot Instance Arbitrage 2026 — https://www.spheron.network/blog/gpu-spot-instance-arbitrage-2026/
Thunder Compute · Cloud GPU Spot Instance Availability and Interruption Rates — https://www.thundercompute.com/blog/cloud-gpu-spot-instance-availability
Synergy Research Group · Neocloud Market Forecast to Approach 400B USD by 2031 — https://www.srgresearch.com/articles/neocloud-market-forecast-to-approach-400b-by-2031-driven-by-surging-ai-infrastructure-demand
Signisys · The Neocloud Revolution: How 20 Billion USD in GPU Providers Reshapes Cloud Market — https://www.signisys.com/blog/the-neocloud-revolution-how-20-billion-in-gpu-focused-providers-are-reshaping-the-cloud-market/
SemiAnalysis · The Great GPU Shortage: Rental Capacity (neocloud taxonomy original) — https://newsletter.semianalysis.com/p/the-great-gpu-shortage-rental-capacity
Spheron Blog · LLM Inference On-Premise vs GPU Cloud: 2026 Cost and Break-Even — https://www.spheron.network/blog/llm-inference-on-premise-vs-cloud/
IntuitionLabs · H100 Rental Prices Compared 1,49-6,98 USD/hr Across 15+ Cloud Providers (2026) — https://intuitionlabs.ai/articles/h100-rental-prices-cloud-comparison

Lambda on lo0 — Tech Blog