Tco on lo0 — Tech Blog

Energy in TCO and in regulation: cost, PUE, CSRD and the new European reporting obligation for datacenters

Tue, 16 Jun 2026 10:00:00 +0200

Notation: amounts in N € or N USD; decimals with a point; thousands separated by a comma. The dollar symbol is not used (it is a formula delimiter). Generic example hardware (4×H100 SXM node, 3.6 kW IT). No real infrastructure.

TL;DR

A 4×H100 SXM node at Spanish industrial prices (0.116 €/kWh, PUE 1.54) spends ~5,475 € on energy per year: 6–8 % of total opex when staff and maintenance dominate, but up to 18–22 % of opex in clusters with a solar PPA and cheap colocation. Energy is not the main TCO lever; it is the only TCO variable that European regulation requires you to measure, publish and certify.

Since September 2024, every datacenter with IT demand ≥ 500 kW in the EU is obliged to report PUE, WUE, ERF and REF to the ReportENER database (Delegated Regulation EU 2024/1364). From August 2027 an automatic sustainability label will be generated from that data. Companies subject to CSRD have to publish Scope 1, 2 and 3 in their sustainability report, along with emissions intensity, which turns the gCO₂/token ratio into a corporate accounting figure, not just an engineering one.

Track context

This article is C8 of the energy pillar. The previous ones:

The full TCO model that this article extends is in:

For the context of technical security compliance:

Technical controls: ENS, ISO 42001 and the EU AI Act

Energy as a TCO component

The reference model

The full TCO model of the on-premise cluster is developed in A8. The reference node: 4×H100 SXM5 80 GB, IT draw ~3.6 kW, 8-node cluster, 3-year amortisation.

Annual opex per node in the base scenario (8-node cluster, Spain):

Opex item	€/year per node	% of total opex
Staff (0.5 FTE pro rata)	7,500	26.8 %
Maintenance / support / failures	9,000	32.1 %
Rack colocation (Spain, high density)	6,000	21.4 %
Energy (PUE 1.54; 0.116 €/kWh)	5,475	19.6 %
Total opex	~28,000	100 %

The annual energy cost formula

$$\text{annual energy cost} = P_{\text{IT}} \times \text{PUE} \times 8{,}760\,\text{h} \times p_{\text{kWh}}$$

where $P_{\text{IT}}$ is the power of the IT equipment (kW), PUE the infrastructure multiplier, and $p_{\text{kWh}}$ the price of electricity (€/kWh).

For the reference node:

$$\text{energy/year} = 3.6\,\text{kW} \times 1.54 \times 8{,}760\,\text{h} \times 0.116\,\frac{\text{EUR}}{\text{kWh}} \approx 5{,}475\,\text{EUR}$$

Sensitivity to energy price and to PUE

Variation in the price of energy has a moderate impact on total TCO when hardware capex dominates:

Energy price (€/kWh)	Energy/year (€)	% opex	€/GPU-hour (70 % util., base)	Var. vs base
0.034 (Spanish solar PPA Q3 2025)	1,604	6.8 %	3.00	−12 %
0.116 (Spanish industrial, base)	5,475	19.6 %	3.39	reference
0.160 (European industrial average)	7,550	25.4 %	3.54	+4 %
0.200 (high tariff, no PPA)	9,437	30.2 %	3.67	+8 %

Variation in PUE is also moderate in total TCO but relevant in the absolute cost of energy and in regulatory reporting:

PUE	Energy/year (€, 0.116 €/kWh)	% opex	€/GPU-hour (70 % util.)
1.15 (liquid cooling)	2,166	8.4 %	3.21
1.20 (modern datacenter)	2,259	8.7 %	3.23
1.48 (datacenters <5 years, Uptime 2025)	3,490	13.1 %	3.33
1.54 (global average, Uptime 2025)	5,475	19.6 %	3.39
1.80 (legacy colocation)	6,400	23.2 %	3.47

The difference between PUE 1.15 and PUE 1.80 in €/GPU-hour is only +8 %, because capex still dominates, but in absolute energy consumed the difference is ×2.96: 2,166 € against 6,400 € a year per node. That difference in absolute energy is exactly what the regulatory framework is going to measure, publish and, in the future, classify.

Energy as a fraction of total TCO

Annual TCO per node includes amortised capex + opex:

$$\text{annual TCO per node} = \frac{\text{capex}}{\text{years}} + \text{annual opex} = 55{,}300\,\text{EUR} + 28{,}000\,\text{EUR} = 83{,}300\,\text{EUR}$$

With these numbers, energy represents:

$$\frac{5{,}475}{83{,}300} \approx 6.6\,\%\ \text{of the all-in TCO}$$

In the low-opex scenario (solar PPA, own datacenter, no colocation):

$$\frac{1{,}604}{68{,}300} \approx 2.3\,\%\ \text{of the TCO}$$

In the high-opex scenario (open-market tariff, expensive colocation):

$$\frac{9{,}437}{130{,}300} \approx 7.2\,\%\ \text{of the TCO}$$

The weight of energy in all-in TCO ranges from 2 % to 7 %. The more capex dominates, the lower the weight of energy. Energy matters more in TCO the longer the amortisation and the lower the relative capex, that is, in scenarios of cheaper hardware or long amortisation periods.

European regulatory framework 2024-2026

Map of the rules

The European framework on datacenters and energy is built from four regulatory layers with different scopes of application, required metrics and effective dates:

Rule	Legal basis	Main obligation	Who it applies to	From when
EED 2023/1791, Art. 12	Directive (EU) 2023/1791	Annual reporting of energy and sustainability KPIs to an EU database	Datacenters ≥ 500 kW IT in the EU	15 Sep. 2024 (1st report)
Delegated Reg. 2024/1364	Art. 33(3) EED	Specifies KPIs, formats and reporting deadlines (Phase 1 of the assessment scheme)	Datacenters ≥ 500 kW IT in the EU	Published 17 May 2024; in force
EU rating scheme (Phase 2, 2026 draft)	Art. 33(3) EED	Automatic sustainability label generated by the ReportENER database	Datacenters that report under 2024/1364	Automatic label from 15 Aug. 2027
CSRD / ESRS E1	Directive 2022/2464/EU + Delegated Regulation (EU) 2023/2772	Sustainability report: Scope 1-2-3 emissions, emissions intensity, transition plan	Large EU companies (>500 employees, from FY 2024); expanding progressively	Financial years from 1 Jan. 2024 (first reports 2025); listed SMEs from 2026
PNIEC 2023-2030 (Spain)	National Integrated Energy and Climate Plan	Energy efficiency 44 %, renewables 81 % in electricity; datacenters included in the target; Royal Decree in progress (consultation Sep. 2025)	Spain	2025-2030

Energy Efficiency Directive: EED 2023/1791

Directive (EU) 2023/1791, published in the OJEU of 20 September 2023 (L 231), recasts the earlier Directive 2012/27/EU. Article 12 introduces for the first time a reporting obligation specific to datacenters at European scale.

Threshold of application: datacenters with an electrical power demand of installed IT systems ≥ 500 kW. Member States may lower this threshold (Germany lowered it to 300 kW; France to 100 kW).

Types of datacenter covered: (a) enterprise datacenters, operated by a company solely for its own IT needs; (b) colocation datacenters, where the customer installs and manages its own equipment; (c) co-hosting or multi-tenant datacenters, where the customer accesses shared infrastructure.

What must be reported: 18 key performance indicators (KPIs) that include installed IT demand, total energy consumption, water consumption, reused waste heat, renewable energy consumed, and the four sustainability indicators: PUE, WUE, ERF and REF.

Deadlines:

First report (2023 data): before 15 September 2024.
Second report (2024 data): before 15 May 2025.
Subsequent annual reports: before 15 May each year.

Delegated Regulation (EU) 2024/1364

Commission Delegated Regulation (EU) 2024/1364 of 14 March 2024 (OJEU L, 17 May 2024) constitutes Phase 1 of establishing the Union’s common assessment regime for datacenters. It specifies:

The performance indicators and the information that operators must report to the European ReportENER database.
The calculation formulas for the four sustainability KPIs (PUE, WUE, ERF, REF).
The reporting format and the classification categories by datacenter type and size.

The Commission has made the ReportENER database available as the single entry point for reporting, or via the equivalent national systems of each Member State.

Phase 2: rating scheme and sustainability label

In March 2026 the European Commission published the public consultation on the draft second Delegated Regulation on the datacenter sustainability rating regime, Phase 2 of the scheme. The consultation was open until 23 April 2026.

The draft establishes that the ReportENER database will automatically generate an electronic sustainability label for every datacenter that has reported under Regulation 2024/1364. The label will show:

PUE and WUE classes based on indices calculated under Annex I of the draft.
REF (renewable energy factor) and ERF (energy reuse factor).
Information on grid functions (demand response capability).

Date of first issue of labels: 15 August 2027, and every year thereafter.

The draft also introduces amendments to Regulation 2024/1364, among them: the possibility of voluntary reporting for datacenters <500 kW, the removal of data traffic indicators from the list of mandatory KPIs, and the relaxation of reporting obligations for colocation operators (reflecting the difficulty of obtaining tenant data for confidentiality reasons).

The Commission’s technical report of October 2025 anticipates that the minimum performance standards (MPS) will be finalised between late 2027 and early 2028, and that they will progressively become mandatory for new builds and significant refurbishments.

CSRD and ESRS E1: corporate emissions reporting

The Corporate Sustainability Reporting Directive (CSRD, Directive 2022/2464/EU) requires companies in scope to publish a sustainability report drawn up under the European Sustainability Reporting Standards (ESRS). The ESRS E1 standard covers climate change.

Temporal scope of application:

Large companies (>500 employees, already subject to NFRD): financial year 2024 data, report published in 2025.
Other large companies (>250 employees or >40 M € turnover or >20 M € balance sheet): financial year 2025 data, report in 2026.
Listed SMEs: financial year 2026 data, report in 2027 (with the possibility of an extension).

What ESRS E1 requires in relation to datacenters and energy:

E1-4: GHG emission reduction targets in absolute value and in intensity (e.g. gCO₂eq per unit of product or service).
E1-5: Energy consumption and mix (renewable vs non-renewable energy, energy intensity).
E1-6: Gross GHG emissions Scope 1, Scope 2 (location-based and market-based) and Scope 3, without netting off offsets; emissions intensity.
E1-7: Removals and carbon projects (relevant for offsets that do not replace reduction).

The amended version of ESRS E1 (EFRAG Exposure Draft of July 2025, submitted to the European Commission in November 2025) expands the disclosure requirements from 9 to 11 and is expected to apply from January 2027.

Relevance for datacenter operators: a company that runs an on-premise GPU cluster for its own AI workloads has those datacenters within the perimeter of its Scope 1 (if it has its own generators) and Scope 2 (purchased electricity). If it contracts cloud services or colocation, those datacenters fall into its Scope 3 (category 1 or 11 under the GHG Protocol).

Spain: PNIEC context and the Royal Decree in progress

Spain’s National Integrated Energy and Climate Plan (PNIEC) 2023-2030, updated in September 2024 (MITECO), sets targets of 81 % renewables in electricity and 44 % energy efficiency for 2030. Datacenters are included among the sectors under intensive efficiency monitoring.

In August-September 2025, MITECO opened the public consultation on the draft Royal Decree regulating the energy efficiency and sustainability of datacenters, transposing it from Art. 12 of EED 2023/1791. The Royal Decree introduces additional requirements for datacenters with total installed rated power above 1 MW: an obligation to use the waste heat generated to supply climate control services or other heat applications.

The deadline for incorporating Art. 12 of the EED expired on 11 October 2025. The Royal Decree is awaiting the opinion of the Council of State.

Regulatory metrics: definitions and formulas

PUE — Power Usage Effectiveness

Defined in ISO/IEC 30134-2:2018 (European equivalent EN 50600-4-2). It is the central metric of the regulatory framework:

$$\text{PUE} = \frac{E_{\text{DC}}}{E_{\text{IT}}}$$

where $E_{\text{DC}}$ is the total energy consumed by the datacenter (including cooling, electrical distribution, lighting and losses) and $E_{\text{IT}}$ is the energy consumed by the IT equipment (servers, storage, network).

PUE = 1.0: theoretical perfection (all the energy goes to compute).
PUE = 1.54: 2025 global average (Uptime Institute, 15th annual survey).
PUE = 1.10–1.15: hyperscaler range.
PUE ≥ 1.80: legacy, unoptimised facilities.

Regulation 2024/1364 requires annual PUE to be measured and reported. The Phase 2 draft introduces PUE classes (A to D or similar) for the sustainability label.

WUE — Water Usage Effectiveness

Defined in ISO/IEC 30134-9:2022:

$$\text{WUE} = \frac{W_{\text{in}}}{E_{\text{IT}}}$$

where $W_{\text{in}}$ is the datacenter’s total water consumption (m³), and the result is expressed in m³/MWh. ReportENER data (2024-2025) shows EU average values between 0.07 and 1.28 m³/MWh depending on Member State, with an EU average of ~0.58 m³/MWh.

WUE is particularly relevant in water-stressed regions, such as southern and inland Spain, where evaporative cooling consumes drinking water.

ERF — Energy Reuse Factor

Defined in ISO/IEC 30134-6:2021:

$$\text{ERF} = \frac{E_{\text{reuse}}}{E_{\text{DC}}}$$

where $E_{\text{reuse}}$ is the energy (waste heat) exported and reused outside the datacenter (e.g. district heating networks, industrial processes). ERF varies between 0 (no reuse) and 1 (all the energy reused, theoretically). The Phase 2 draft proposes minimum ERF thresholds for new facilities.

REF — Renewable Energy Factor

$$\text{REF} = \frac{E_{\text{res,tot}}}{E_{\text{DC}}}$$

where $E_{\text{res,tot}}$ is the total renewable energy consumed (own generation + PPAs + Guarantees of Origin), divided by the datacenter’s total consumption. The Commission’s proposal points to targets of 75–100 % REF by 2030.

CUE — Carbon Usage Effectiveness

Defined in ISO/IEC 30134-8:2022. Not included in the reporting obligation of Regulation 2024/1364, but identified as a future KPI in the Commission’s technical report (Oct. 2025):

$$\text{CUE} = \frac{G_{\text{DC}}}{E_{\text{IT}}}$$

where $G_{\text{DC}}$ is the datacenter’s total GHG emissions (gCO₂eq) and the result is expressed in gCO₂eq/kWh IT. It is equivalent to the product $\text{PUE} \times I_{\text{grid}}$ when only location-based Scope 2 emissions are considered.

Master table: rule × what it requires × metric/threshold × date

Rule	What it requires	Metrics / thresholds	Date of application	To whom
EED 2023/1791, Art. 12	Annual reporting of KPIs to an EU database (ReportENER)	PUE, WUE, ERF, REF + 14 additional KPIs (energy, water, waste heat, renewables)	15 Sep. 2024 (1st report, 2023 data); 15 May annually from 2025	Datacenters ≥ 500 kW IT in the EU
Delegated Reg. (EU) 2024/1364	Technical specification of KPIs and reporting format (Phase 1 of the scheme)	PUE = E DC/E IT; WUE = W in/E IT; ERF = E reuse/E DC; REF = E res/E DC	In force since 17 May 2024	Datacenters ≥ 500 kW IT in the EU
EU rating scheme, Phase 2 (2026 draft)	Automatic electronic sustainability label with PUE and WUE classes	PUE and WUE classes A-D (thresholds in Annex I of the draft; pending final adoption)	Automatic labels from 15 Aug. 2027	Datacenters that have reported under 2024/1364
Minimum performance standards (MPS, expected 2027-2028)	Mandatory minimum PUE, REF and ERF thresholds for new builds and refurbishments	PUE targets differentiated by climate and size; REF ≥ 75-100 % by 2030; minimum ERF for new facilities	Finalisation expected 2027-2028; progressively mandatory	New and refurbished datacenters
CSRD / ESRS E1	Sustainability report: GHG emissions Scope 1, 2 (location- and market-based) and 3; emissions intensity; transition plan	Scope 1+2+3 mandatory in tCO₂eq; intensity in gCO₂eq/functional unit	Data from 1 Jan. 2024 (>500 employees); from 1 Jan. 2025 (>250 employees)	Large EU companies subject to CSRD
PNIEC 2023-2030 / Spanish Royal Decree (in progress)	Energy efficiency and sustainability of datacenters; obligation to use waste heat	Renewables target 81 % by 2030; datacenters ≥ 1 MW: waste heat for climate control or other applications	Royal Decree pending approval (consultation Sep. 2025)	Datacenters in Spain

How Wh/token and gCO₂/token connect with the required reporting

The conversion chain

The technical inference efficiency figure, J/token or Wh/token, measured with tools such as DCGM, NVML or nvidia-smi, becomes a regulatory reporting figure through the following chain:

$$E_{\text{token}} = \frac{P_{\text{IT}}}{T} \quad \text{[Wh/tok]}$$ $$E_{\text{DC/token}} = E_{\text{token}} \times \text{PUE} \quad \text{[Wh/tok adjusted to the datacenter]}$$ $$\text{gCO}_{2}\text{/token} = E_{\text{DC/token}} \times I_{\text{grid}} \quad \text{[gCO}_2\text{eq/tok]}$$

where $T$ is throughput (tok/s), $P_{\text{IT}}$ the power of the IT node, and $I_{\text{grid}}$ the carbon intensity of the grid (gCO₂eq/kWh). The full methodology is in C6.

For the reference node (4×H100 SXM, $P_{\text{IT}} = 3.6\,\text{kW}$, indicative throughput 1,000 tok/s, PUE 1.54):

$$E_{\text{DC/token}} = \frac{3.6\,\text{kW} \times 1.54}{1{,}000\,\text{tok/s}} = 5.544\,\text{J/tok} = 1.54 \times 10^{-3}\,\text{Wh/tok}$$

From Wh/token to the regulatory KPI: aggregation

What regulatory reporting (Regulation 2024/1364) requires is not per-token ratios but annual totals:

$E_{\text{IT}}$: total energy of the IT equipment over the year (kWh).
$E_{\text{DC}}$: total datacenter energy over the year (kWh), including cooling and distribution.
$W_{\text{in}}$: total water consumption (m³).
$E_{\text{reuse}}$: energy (waste heat) exported and reused (kWh).
$E_{\text{res,tot}}$: renewable energy consumed (kWh).

The four sustainability KPIs are calculated from those totals. The link between the per-token ratio and regulatory reporting is:

$$E_{\text{IT,annual}} = \sum_{\text{inferences}} E_{\text{token}} \times N_{\text{tokens}} + E_{\text{idle}} \quad \text{[kWh/year]}$$

The $E_{\text{token}}$ figure is needed in order to attribute total IT consumption to inference workloads (vs idle, vs fine-tuning), which allows the energy intensity of inference to be calculated as a fraction of the datacenter’s total consumption, a figure relevant to ESRS E1 reporting of emissions intensity.

From Wh/token to gCO₂/token as a CSRD accounting figure

Under ESRS E1-6, companies subject to CSRD must report GHG emissions with intensity per functional unit (E1-4). For a company running an on-premise LLM, the natural functional unit is the token generated, the query answered or the user served (under the Green Software Foundation’s SCI specification, ISO/IEC 21031:2022).

gCO₂/token becomes a corporate accounting figure like this:

Technical measurement: J/token per node (DCGM/NVML).
PUE adjustment: Wh/token at datacenter level = J/token × PUE / 3,600.
Carbon intensity: gCO₂/token = Wh/token × $I_{\text{grid}}$ / 1,000.
Annual scaling: tCO₂eq/year = gCO₂/token × annual N tokens / 10⁶ / 10⁶.
ESRS E1-6 reporting: location-based Scope 2 emissions (with national annual average $I_{\text{grid}}$) and market-based (with $I_{\text{grid}}$ according to renewable energy contracts), in tCO₂eq/year + intensity in gCO₂eq/token.

The location-based / market-based distinction from the GHG Protocol Scope 2 Guidance is mandatory under ESRS E1-6: both methods must be reported.

Worked example: from J/token to tCO₂eq/year

Reference node (4×H100 SXM, PUE 1.54, throughput 1,000 tok/s, 70 % utilisation, Spanish average grid intensity 2024 ~108 gCO₂/kWh):

$$E_{\text{DC/token}} = 1.54 \times 10^{-3}\,\text{Wh/tok}$$ $$\text{gCO}_{2}\text{/tok} = 1.54 \times 10^{-3} \times 108 \approx 0.166\,\text{gCO}_{2}\text{eq/tok}$$

Tokens generated per year (1,000 tok/s × 70 % util. × 8,760 h × 3,600 s/h):

$$N_{\text{tokens/year}} = 1{,}000 \times 0.70 \times 8{,}760 \times 3{,}600 \approx 2.2 \times 10^{10}\,\text{tok}$$

Annual location-based Scope 2 emissions per node:

$$\text{tCO}_{2}\text{eq/year} = \frac{0.166 \times 2.2 \times 10^{10}}{10^{12}} \approx 3.65\,\text{tCO}_{2}\text{eq}$$

For an 8-node cluster: ~29.2 tCO₂eq/year. With a solar PPA (effective intensity ~20 gCO₂/kWh market-based): ~5.4 tCO₂eq/year. The difference between the two methods, location-based vs market-based, is ×5.4, and both have to appear in the CSRD report.

Scenario	Grid intensity (gCO₂/kWh)	gCO₂/tok	tCO₂eq/year (1 node, 70 % util.)	ESRS method
Spanish average 2024 (location-based)	108	0.166	3.65	Location-based mandatory
Spanish solar PPA (market-based)	~20	0.031	0.68	Market-based mandatory
French average 2025 (location-based)	19.6	0.030	0.66	Location-based mandatory
German average 2025 (location-based)	328	0.505	11.1	Location-based mandatory

The 500 kW threshold and on-premise AI datacenters

Which datacenters does it affect in practice?

A 4×H100 SXM5 node draws ~3.6 kW IT. The 500 kW IT threshold is equivalent to ~139 nodes of 4×H100 SXM5. For a smaller cluster, the regulatory threshold looks distant.

There are, however, three routes by which it applies to smaller on-premise AI infrastructure:

Route	Explanation
Shared datacenter	If the GPU nodes sit in a larger corporate datacenter that exceeds 500 kW IT in total (not only GPUs), the datacenter as a whole is subject to reporting, and it is the datacenter operator who reports. The end user receives the PUE of its facility.
Threshold lowered by a Member State	France lowered the threshold to 100 kW IT (~28 4×H100 nodes). Germany set it at 300 kW (~83 nodes). In those countries, medium-sized AI clusters are already subject to reporting.
CSRD independent of the datacenter threshold	The CSRD obligation applies to the company according to its size, not to the datacenter. A company subject to CSRD that operates even a single GPU node is obliged to include the Scope 2 emissions of that node in its sustainability report.

PUE as an operational figure and as a regulatory figure

PUE is simultaneously:

An operational figure for the TCO model (multiplier of energy cost and of gCO₂/token).
A regulatory KPI subject to mandatory reporting under Regulation 2024/1364.
A classification criterion in the future Phase 2 sustainability label.

The reference technical standard for measurement is ISO/IEC 30134-2:2018, which defines four levels of PUE measurement depending on where IT power is measured (from the rack to the incoming transformers). Regulation 2024/1364 specifies that PUE measurement for reporting to ReportENER must follow this standard.

For a typical on-premise facility, PUE has not historically been measured with the granularity the standard demands. The first compliance step is to instrument the datacenter with power meters at the points specified by ISO/IEC 30134-2.

Sensitivity of gCO₂/token to PUE and energy price

The following table shows the variation in gCO₂/token for the reference node (4×H100 SXM, 1,000 tok/s, Spain) as a function of PUE and of the energy scenario (which determines the effective carbon intensity):

PUE	Grid intensity (gCO₂/kWh)	gCO₂/1,000 tok	gCO₂/1M tok	Var. vs base
1.15 (liquid, Spain solar)	50	0.064	63.9	−55 %
1.15 (liquid, Spain average)	108	0.138	138.0	−2 %
1.54 (global average, Spain average)	108	0.166	166.3	reference
1.54 (global average, Spain night gas)	220	0.339	338.8	+104 %
1.80 (legacy, Spain average)	108	0.194	194.4	+17 %
1.80 (legacy, Spain night gas)	220	0.396	396.0	+138 %

Grid intensity has more impact than PUE on gCO₂/token: going from Spanish solar (50 gCO₂/kWh) to Spanish night-time gas (220 gCO₂/kWh) multiplies emissions by ×4.4, while going from PUE 1.15 to 1.80 multiplies them only by ×1.57. The combination of a dirty grid plus a high PUE is the worst: ×6.2 against the base case.

For location-based CSRD reporting, the intensity used is the national annual average (or regional if more granular data exists), not the hourly figure. For market-based reporting, the effective intensity depends on the energy contract (PPA, Guarantees of Origin). Carbon-aware temporal shifting (moving load to hours of lower intensity) reduces real emissions but does not directly change location-based reporting, which uses the annual grid average.

Connection between operator metrics and regulator obligations

The map of required data

Technical figure	How it is obtained	Use in TCO	Use in regulatory reporting
$P_{\text{IT}}$ (W)	DCGM / NVML / IPMI	Energy cost formula	$E_{\text{IT,annual}}$ for PUE, WUE, ERF, REF
Throughput (tok/s)	Engine metrics (vLLM, TGI)	€/token, Wh/token	Energy intensity of inference
PUE (measured)	Meters on PDUs and IT equipment (ISO/IEC 30134-2)	Multiplier in the TCO formula	Mandatory ReportENER KPI
Annual $I_{\text{grid}}$ (gCO₂/kWh)	REE / MITECO / national factors	gCO₂/token for carbon-aware	Emission factor for location-based Scope 2 ESRS E1-6
Hourly $I_{\text{grid}}$	ElectricityMaps / Carbon Aware SDK	Carbon-aware scheduling	Location-based Scope 2 with hourly granularity (best practice)
$W_{\text{in}}$ (m³)	Datacenter water meters	Cost of water (where applicable)	WUE for ReportENER
$E_{\text{reuse}}$ (kWh)	Heat export meters	Possible waste heat revenue	ERF for ReportENER
$E_{\text{res,tot}}$ (kWh)	Energy bills with a renewable attribute / GOs	Energy cost with a PPA	REF for ReportENER; market-based Scope 2 for ESRS E1-6
Hardware embodied carbon (gCO₂eq)	Supplier LCA (AWS methodology, where available)	—	Scope 3 cat. 1 for ESRS E1-6

The minimum measurement system for compliance

For an on-premise datacenter subject to the obligations of Regulation 2024/1364 and to CSRD, the minimum measurement system needed is:

A power meter at the datacenter entry point (total datacenter energy, $E_{\text{DC}}$).
Power meters on the IT racks (IT equipment energy, $E_{\text{IT}}$): in practice, metered PDUs or the sum of DCGM/NVML measurements.
A water meter for the datacenter cooling circuit ($W_{\text{in}}$).
A waste heat export meter if the datacenter has a recovery system ($E_{\text{reuse}}$).
A record of energy bills with a breakdown of renewable energy (PPAs, GOs) for calculating REF and market-based Scope 2.

Without this measurement system, reporting to ReportENER cannot be correct and CSRD reporting lacks the data base it needs.

Minimum Performance Standards: the 2027-2028 outlook

The European Commission’s technical report (Assessment, July 2025, published on the EU Op Portal) and EUDCA’s monitoring anticipate that the Minimum Performance Standards (MPS) for datacenters will be defined in the 2027-2028 period with the following expected characteristics (not finalised):

KPI	MPS proposed for new builds	MPS proposed for existing facilities	Horizon
PUE	≤ 1.3 (cold climate); ≤ 1.4 (temperate/warm climate)	Staged targets differentiated by age	2028+
REF	≥ 75 % renewables in 2025; ≥ 100 % by 2030	Progressive, following the CNDCP roadmap	2025-2030
ERF	Minimum threshold for new facilities with usable heat output	Feasibility assessment requirement	2028+
WUE	No general threshold proposed yet; focus on potable vs non-potable water	Monitoring and reporting	2027+

These values are indicative and come from the consultation process; the definitive values will appear in the second or third Delegated Regulation adopted under Art. 33(3) of the EED.

Sources

EUR-Lex · Directiva (UE) 2023/1791 del Parlamento Europeo y del Consejo, de 13 de septiembre de 2023, relativa a la eficiencia energética (EED refundida) — https://eur-lex.europa.eu/legal-content/ES/TXT/?uri=OJ:JOL_2023_231_R_0001
EUR-Lex · Reglamento Delegado (UE) 2024/1364 de la Comisión, de 14 de marzo de 2024, relativo a la primera fase del establecimiento de un régimen de evaluación común de la Unión para centros de datos — https://eur-lex.europa.eu/eli/reg_del/2024/1364/oj/eng
BOE / DOUE · Texto en español del Reglamento Delegado (UE) 2024/1364 — https://www.boe.es/buscar/doc.php?id=DOUE-L-2024-80715
Comisión Europea · Commission adopts EU-wide scheme rating sustainability of data centres (15 mar. 2024) — https://energy.ec.europa.eu/news/commission-adopts-eu-wide-scheme-rating-sustainability-data-centres-2024-03-15_en
Comisión Europea · Rating scheme for data centres in the EU: Commission launches call for feedback (26 mar. 2026) — https://energy.ec.europa.eu/news/rating-scheme-data-centres-eu-commission-launches-call-feedback-2026-03-27_en
DLA Piper · EU Commission proposes new data centre sustainability rating label (20 abr. 2026) — https://www.dlapiper.com/en-us/insights/blogs/environment-health-safety-and-product-compliance/2026/eu-commission-proposes-new-data-centre-sustainability-rating-label
RE24 Energy · What EU New Data Centre Rating Scheme Means for Operators (dic. 2025) — https://re24.energy/blog-what-eu-new-data-centre-rating-scheme-means/
Comisión Europea · Assessment of next steps to promote the energy performance and sustainability of data centres in the EU (2nd Technical Report, jul. 2025) — https://op.europa.eu/o/opportal-service/download-handler?identifier=83be4c3e-5c79-11f0-a9d0-01aa75ed71a1&format=PDF&language=en&productionSystem=cellar
Comisión Europea · Assessment of next steps — October 2025 follow-up — https://op.europa.eu/o/opportal-service/download-handler?identifier=29aa14ed-a4ba-11f0-a7c5-01aa75ed71a1&format=PDF&language=en&productionSystem=cellar
Comisión Europea · Minimum performance standards for EU data centres (preparatory study) — https://energy.ec.europa.eu/resources/preparatory-studies/minimum-performance-standards-eu-data-centres_en
MITECO · Centros de datos: normativa y reporte (España) — https://www.miteco.gob.es/en/energia/eficiencia/centros-de-datos/normativa.html
MITECO · PNIEC 2023-2030 (actualización sept. 2024) — https://www.miteco.gob.es/content/dam/miteco/es/energia/files-1/pniec-2023-2030/PNIEC_2024_240924.pdf
Ashurst · Draft Royal Decree regulating energy efficiency and sustainability of data centres in Spain (sept. 2025) — https://www.ashurst.com/es-es/insights/draft-royal-decree-regulating-energy-efficiency-and-sustainability-of-data-centres-in-spain/
EFRAG · Amended ESRS E1 Exposure Draft (jul. 2025) — https://www.efrag.org/sites/default/files/media/document/2025-07/Amended_ESRS_Exposure_Draft_July_2025_ESRS_E1_0.pdf
Normative · ESRS E1 Explained: CSRD Climate Disclosure (2026) — https://normative.io/insight/esrs-e1/
CSRD Pro · Reporting Cloud Services and Data Centres for CSRD — https://www.csrdpro.com/en/report-cloud-services-data-centres-csrd/
Uptime Institute · Global Data Center Survey 2025 (PUE medio global 1,54) — https://datacenter.uptimeinstitute.com/rs/711-RIA-145/images/2025.Annual.Survey.Report.pdf
SOCOMEC · Energy Efficiency Directive: Essential Guidelines for Data Center Compliance — https://emea.socomec.com/en/news/eu-energy-efficiency-directive-what-new-rules-mean-data-centres
ISO/IEC 30134-2:2018 · Power Usage Effectiveness (PUE) — https://www.iso.org/standard/63451.html
ISO/IEC 30134-6:2021 · Energy Reuse Factor (ERF) — https://www.iso.org/standard/75728.html
ISO/IEC 30134-8:2022 · Carbon Usage Effectiveness (CUE) — https://www.iso.org/standard/82562.html
ISO/IEC 30134-9:2022 · Water Usage Effectiveness (WUE) — https://www.iso.org/standard/82563.html
Green Software Foundation · Software Carbon Intensity (SCI) Specification (ISO/IEC 21031:2022) — https://sci.greensoftware.foundation/
White & Case · Data centres and energy consumption: evolving EU regulatory landscape and outlook for 2026 — https://www.whitecase.com/insight-alert/data-centres-and-energy-consumption-evolving-eu-regulatory-landscape-and-outlook-2026
Datacenter Forum · New reporting requirements for data center operators introduced by the Delegated Regulation (EU) 2024/1364 — https://datacenter-forum.ro/en/new-reporting-requirements-for-data-center-operators-introduced-by-the-delegated-regulation-eu-2024-1364/

Full TCO of an on-premise GPU cluster: from capex to the all-in €/GPU-hour and the break-even against cloud

Tue, 16 Jun 2026 06:00:00 +0200

Notation: amounts in N € or N USD (source denominated in dollars); decimal point; comma as thousands separator. The dollar sign is not used (it is the formula delimiter). Data centred on Europe/Spain. Generic example hardware: a cluster of N nodes, each with 4×H100 SXM5 80 GB.

TL;DR

A 4×H100 SXM5 node costs between 150,000 USD and 200,000 USD in total capex (GPUs + server + network + storage + prorated rack). Amortised over 3 years with European opex (energy at ~0.116 €/kWh industrial, PUE 1.54 average or 1.2 with liquid, 0.3 FTE of staff), the all-in cost ranges from 3.10 USD/GPU-hour (100 % utilisation) to 6.20 USD/GPU-hour (50 % utilisation). The break-even against AWS p5 on-demand (~6.88 USD/GPU-hour) is crossed at around 70 % utilisation; against a neocloud 3-year reserved (~1.49–2.10 USD/GPU-hour), on-prem never closes the gap in that scenario. Utilisation is the variable that decides the cost axis, not the price of the hardware.

The model: declared assumptions

Every calculation below starts from these assumptions. Changing any of them moves the conclusion; the sensitivity section quantifies by how much.

Parameter	Base value	Sensitivity range
Node	4×H100 SXM5 80 GB (HGX baseboard)	—
Cluster	N nodes (per-node model; scales linearly)	1–32 nodes
Capex amortisation	3 years (straight line)	3–5 years
GPU utilisation	70 %	30 %–100 %
Energy price	0.116 €/kWh (industrial Spain, Sept. 2025)	0.06–0.20 €/kWh
PUE	1.54 (global average, Uptime Institute 2025)	1.15–1.80
Exchange rate	1 USD = 0.93 € (reference Jun. 2026)	—

Energy source: GlobalPetrolPrices · Spain Business Electricity, Sept. 2025. PUE source: Uptime Institute Global Data Center Survey 2025 — the global average PUE has been stuck at 1.54 for the sixth consecutive year; hyperscalers 1.10–1.15; colocation/enterprise 1.58–1.80; facilities less than 5 years old, 1.48. PUE 1.2 is achievable with direct-to-chip liquid cooling.

Capex breakdown per 4×H100 SXM5 node

GPUs

Component	Unit price (USD)	Quantity	Subtotal (USD)	Source and date
H100 SXM5 80 GB (card)	30,000–40,000	4	120,000–160,000	GMI Cloud, Apr. 2026 · Introl, Apr. 2026

The range reflects market variability and volume discounts (5–15 % for orders above 50 units). H100 SXM5 cards require NVIDIA’s HGX baseboard; they are not sold loose for direct installation in standard servers.

Server / HGX baseboard

A complete 4×H100 SXM node uses NVIDIA’s HGX H100 4-GPU baseboard plus a compatible host server. Reference models: Supermicro SYS-421GU-TNXR (4U, dual Intel Xeon 4th Gen, HGX H100 4-GPU) and its Dell equivalent.

Component	Estimated cost (USD)	Note
Server chassis + CPU (2× Xeon) + RAM (512 GB DDR5) + redundant PSU	18,000–25,000	Based on a Supermicro SYS-821GE bare without GPUs at ~24,806 USD (xicomputer.com, Jun. 2026); scaled to the 4-GPU
HGX H100 4-GPU baseboard	included in the GPU price	NVIDIA HGX platform; no separate public price
Inter-GPU NVLink (inside the node)	included in the baseboard	4 GPUs connected by NVLink 4.0 on the HGX baseboard

Marketing claim (no independent verification): Supermicro announces datacenter energy cost reductions of up to 40 % with liquid cooling in its HGX H100 servers (Supermicro press release).

InfiniBand NDR network

For a multi-node cluster with tensor parallelism across nodes, the GPU-to-GPU network is critical. NDR InfiniBand (400 Gb/s per port) is the de facto standard for HGX clusters.

Component	Estimated cost per node (USD)	Source / Note
NVIDIA Quantum-2 NDR 400G switch (64 ports, prorated across N nodes)	2,000–4,000	Switch ~35,000 USD (Introl, Apr. 2026); at 16 nodes, ~2,200 USD/node
InfiniBand NDR cables/transceivers (4 ports per node × ~1,000 USD/port)	4,000	Estimate based on ~1,000 USD per optical transceiver (Introl, Apr. 2026)
InfiniBand network (prorated per 4-GPU node)	~6,000–8,000	—

For inference serving inside a single node (4 GPUs with NVLink), the inter-node network is less critical than for multi-node training. For disaggregated prefill-decode workloads spread across nodes, InfiniBand NDR is necessary.

NVMe storage

Component	Estimated cost (USD)	Note
Local NVMe (4 TB × 2 U.2/E1.S drives, working datasets and checkpoints)	2,000–4,000	~500–1,000 USD/TB enterprise NVMe, 2025
Shared object storage (NAS/MinIO, prorated per node)	2,000–5,000	Varies with the total capacity of the cluster
Total storage per node	~4,000–9,000	—

Introl models 50 TB per GPU for effective operation in training clusters (Introl, Apr. 2026); for pure inference the requirement is significantly lower (model weights + logs).

Rack, PDU and datacenter connectivity

Component	Estimated cost per node (USD/year)	Source
Rack colocation (high density, 10–15 kW per node)	5,000–12,000/year	Encoradvisors · Colocation Pricing 2026: high density 3,000–6,000 USD/month per rack; at 2 nodes per rack, ~1,500–3,000 USD/month per node = 18,000–36,000 USD/year in tier-1; lower in Spain
Rack PDU, electrical cabling (prorated)	500–1,000 per node (amortised capex)	Inside the colocation line item or in an own datacenter

Colocation in Spain/Europe is structurally cheaper than in US tier-1 markets (New York, Silicon Valley). For an own datacenter, replace it with the cost of your own space plus amortisation of the electrical and cooling infrastructure.

Capex summary per 4×H100 SXM5 node

Line item	Range (USD)	Midpoint
GPUs (4× H100 SXM5)	120,000–160,000	140,000
Server chassis + CPU/RAM/PSU	18,000–25,000	21,500
InfiniBand NDR network (prorated)	6,000–8,000	7,000
NVMe + object storage	4,000–9,000	6,500
PDU/rack/other (capex)	2,000–5,000	3,500
Total capex per node	150,000–207,000	178,500

Sources: GMI Cloud (Apr. 2026), Introl (Apr. 2026), Spheron (Apr. 2026), xicomputer.com (Jun. 2026).

Opex breakdown per 4×H100 SXM5 node (annual)

Energy

A 4×H100 SXM5 node at full load draws approximately:

$$P_{\text{node}} = 4 \times 700\,\text{W (TDP H100 SXM5)} + 800\,\text{W (server)} \approx 3.6\,\text{kW (IT)}$$

The total datacenter power includes the cooling overhead, expressed by the PUE:

$$P_{\text{total}} = P_{\text{IT}} \times \text{PUE}$$ $$\text{annual energy cost} = P_{\text{IT}} \times \text{PUE} \times 8{,}760\,\text{h} \times \text{kWh price}$$

With the base values (PUE 1.54; 0.116 €/kWh):

$$\text{energy/year} = 3.6\,\text{kW} \times 1.54 \times 8{,}760\,\text{h} \times 0.116\,\text{EUR/kWh} \approx 5{,}475\,\text{EUR}$$

With a Spanish solar PPA (Q3 2025 reference price: ~34 €/MWh = 0.034 €/kWh according to PV Tech, Oct. 2025):

$$\text{energy/year (solar PPA)} = 3.6 \times 1.54 \times 8{,}760 \times 0.034 \approx 1{,}604\,\text{EUR}$$

Energy scenario	Price (€/kWh)	Energy cost/year per 4-GPU node
Spanish solar PPA (Q3 2025)	0.034	~1,604 €
Industrial Spain (Sept. 2025)	0.116	~5,475 €
European average (industrial tariff)	0.160	~7,550 €
Worst case (no PPA, high tariff)	0.200	~9,437 €

Staff / operations

Staff cost is the most variable line item with cluster size. For a small cluster (2–8 nodes), the rule of thumb is 0.3–0.5 FTE per cluster of GPU infrastructure support (Spheron, Apr. 2026).

Cluster size	Estimated FTE	FTE cost (€/year, Western Europe)	Cost per 4-GPU node (€/year)
2–4 nodes	0.3 FTE	~120,000	36,000–18,000
8–16 nodes	0.5 FTE	~120,000	7,500
32+ nodes	1–2 FTE	~120,000	3,750–7,500

Indicative salary reference: a GPU infrastructure engineer with CUDA, InfiniBand and Kubernetes knowledge in Western Europe, ~90,000–140,000 €/year fully loaded. Introl’s figures (Apr. 2026) in USD (~275,000 USD/year for the US) reflect the North American market, which is appreciably higher.

Maintenance and support

Line item	Annual cost (% of hardware capex)	Per 4-GPU node (midpoint)
Vendor maintenance / support	5–10 % of capex	~7,000–14,000 USD → ~6,500–13,000 €
GPU failure rate (~5 % annual) × replacement cost	5 % × 4 GPUs × ~35,000 USD = ~7,000 USD expected	~6,500 € (amortised as a provision)
Minor spares (cables, modules)	~500–1,000 €	—

Introl quotes GPU failure rates of 2–3 % per year in small clusters; Google Research documented ~9 % annualised in Meta’s 16,384-GPU H100 cluster (Introl, Apr. 2026). 5 % is used here as a conservative intermediate value.

Depreciation (for accounting purposes)

Straight-line depreciation turns capex into an annual flow comparable to the cost of committed cloud:

$$\text{annual depreciation} = \frac{\text{node capex}}{\text{amortisation years}}$$

Node capex (USD)	3-year amortisation (USD/year)	5-year amortisation (USD/year)
150,000 (minimum)	50,000	30,000
178,500 (average)	59,500	35,700
207,000 (maximum)	69,000	41,400

H100 hardware depreciates fast: secondary-market analyses put the residual value at 20–40 % of the purchase price after 3 years (Introl, Apr. 2026). The arrival of Blackwell GB200/GB300 accelerates the perceived obsolescence.

Annual opex summary per 4×H100 SXM5 node (base scenario, 8-node cluster)

Line item	Base scenario (€/year)	Range
Energy (PUE 1.54; 0.116 €/kWh)	5,475	1,604–9,437
Staff (0.5 FTE × 8 nodes, prorated)	7,500	3,750–36,000
Maintenance / support / failures	9,000	5,000–15,000
Rack colocation (Spain, high density)	6,000	3,000–15,000
Total opex per node	~28,000	~13,000–75,000

The extreme range reflects the difference between a well amortised own datacenter with a solar PPA and cheap energy (minimum opex) and tier-1 colocation with market tariffs and senior staff.

Deriving the all-in €/GPU-hour

Formula

$$\text{EUR/GPU-hour all-in} = \frac{\frac{\text{node capex}}{\text{years}} + \text{node annual opex}}{4\,\text{GPUs} \times 8{,}760\,\text{h} \times u}$$

where $u$ is the average annual utilisation (0 to 1).

See the cost-per-token identity in cost per token and per request for the connection with throughput.

€/GPU-hour table by utilisation and scenario

Average capex (178,500 USD → ~166,000 €), 3-year amortisation → 55,300 €/year.

Utilisation	Opex/year (base, €)	Total cost/year (€)	Useful GPU-hours/year	€/GPU-hour
30 %	28,000	83,300	10,512	7.93
50 %	28,000	83,300	17,520	4.75
70 %	28,000	83,300	24,528	3.39
80 %	28,000	83,300	28,032	2.97
100 %	28,000	83,300	35,040	2.38

Low-opex scenario (solar PPA, own datacenter, large cluster): opex/year ~13,000 €.

Utilisation	Total cost/year (€)	€/GPU-hour
50 %	68,300	3.90
70 %	68,300	2.78
80 %	68,300	2.43
100 %	68,300	1.95

High-opex scenario (market tariff, expensive colocation, small cluster): opex/year ~75,000 €.

Utilisation	Total cost/year (€)	€/GPU-hour
50 %	130,300	7.44
70 %	130,300	5.31
80 %	130,300	4.65
100 %	130,300	3.72

From €/GPU-hour to €/1M tokens

The cost-per-token identity connects the hardware cost with the inference cost:

$$\text{EUR/1M tokens} = \frac{\text{EUR/GPU-hour} \times 10^6}{\text{throughput (tok/s)} \times 3{,}600}$$

For reference throughputs on H100 SXM5 with vLLM (see capacity planning for on-premise inference):

Model	Typical throughput (tok/s per GPU)	Source
Llama-3 70B FP8, high batch	~2,800	Series B benchmarks
Llama-3 8B FP16, medium batch	~9,000	Series B benchmarks
Mixtral 8×7B, high batch	~4,500	Series B benchmarks

€/1M tokens table in the base scenario (€/GPU-hour 3.39 at 70 % utilisation):

Model	Throughput (tok/s)	€/1M tokens
Llama-3 70B FP8	2,800	~0.336
Llama-3 8B FP16	9,000	~0.105
Mixtral 8×7B	4,500	~0.209

At 50 % utilisation (€/GPU-hour 4.75):

Model	€/1M tokens
Llama-3 70B FP8	~0.471
Llama-3 8B FP16	~0.147

Occupancy (batching) multiplies the effective throughput and lowers the €/1M tokens without changing the hardware; it is analysed in GPU utilisation as a FinOps lever.

Break-even on-prem vs cloud

The break-even formula

Break-even happens when the total annual on-prem cost equals the annual cloud cost at the same utilisation:

$$\text{annual cloud cost} = \text{cloud GPU-hour price} \times 4\,\text{GPUs} \times 8{,}760\,\text{h} \times u$$ $$\text{break-even}: \quad \frac{\text{capex/year} + \text{opex/year}}{4 \times 8{,}760 \times u} = \text{cloud GPU-hour price}$$

Solving for the break-even utilisation:

$$u^* = \frac{\text{capex/year} + \text{opex/year}}{4 \times 8{,}760 \times \text{cloud GPU-hour price}}$$

Break-even table by cloud mode and on-prem scenario

Base on-prem scenario (capex/year 55,300 €, opex/year 28,000 €, total 83,300 €/year per 4-GPU node):

Cloud reference (price/GPU-hour)	USD equiv.	Break-even utilisation $u^*$	Note
Neocloud on-demand (Lambda/Spheron ~2.90 USD)	2.90 USD (~2.70 €)	>100 %, on-prem does not compete	Neocloud on-demand is cheaper even at full utilisation
Neocloud 3-year reserved (CoreWeave ~1.49–2.10 USD)	~1.80 USD (~1.67 €)	>100 %, impossible	Neocloud reserved beats on-prem in every scenario of this model
AWS p5 on-demand (6.88 USD/GPU-hour)	6.88 USD (~6.40 €)	~47 %	Above 47 %, the average on-prem beats AWS on-demand
AWS p5 3-year reserved (~2.97 USD/GPU-hour)	2.97 USD (~2.76 €)	>100 %	—
GCP A3 on-demand (~10.98 USD/GPU-hour)	10.98 USD (~10.21 €)	~29 %	Above 29 %, on-prem beats GCP on-demand
Azure ND H100 v5 on-demand (~12.29 USD/GPU-hour)	12.29 USD (~11.43 €)	~26 %	—

Low-opex scenario (total 68,300 €/year):

Cloud reference	Break-even utilisation
AWS p5 on-demand (6.88 USD ≈ 6.40 €)	~38 %
Neocloud on-demand (2.90 USD ≈ 2.70 €)	~91 %
Neocloud 3-yr reserved (1.80 USD ≈ 1.67 €)	>100 %

High-opex scenario (total 130,300 €/year):

Cloud reference	Break-even utilisation
AWS p5 on-demand (6.88 USD ≈ 6.40 €)	~72 %
GCP A3 on-demand (~10.21 €)	~45 %
Azure on-demand (~11.43 €)	~41 %

Reading the break-even table

Against neoclouds (on-demand or reserved), on-prem TCO does not reach break-even in any scenario of the base model. Neocloud reserved beats on-prem even at 100 % utilisation, because its hourly price is below the all-in cost of your own hardware. This is consistent with the analysis in cloud GPU: on-demand, reserved and spot.
Against on-demand hyperscalers (AWS, GCP, Azure), on-prem does have an achievable break-even: around 26–72 % utilisation depending on the scenario. At medium-high utilisation (>70 %), on-prem clearly beats AWS/GCP/Azure on-demand.
The variable that moves the break-even most is opex (staff above all), not hardware capex. A well sized cluster in cheap colocation with PPA energy can lower the threshold by 20 percentage points compared with the high scenario.
For GDPR data, the break-even against US hyperscalers is skewed: the sovereignty axis rules out US hyperscalers before cost does (see sovereign on-premise vs hyperscalers).

Sensitivity analysis

TCO vs utilisation

The all-in cost per GPU-hour varies inversely with utilisation because capex is fixed:

$$\frac{d(\text{EUR/GPU-hour})}{du} = -\frac{\text{capex/year} + \text{opex/year}}{4 \times 8{,}760 \times u^2} < 0$$

Going from 50 % to 80 % utilisation cuts the €/GPU-hour by $\frac{4.75 - 2.97}{4.75} \approx 37\,\%$ in the base scenario. That 37 % reduction requires no hardware change, only more efficient scheduling (see GPU utilisation as a FinOps lever).

Utilisation	€/GPU-hour (base scenario)	Change vs 50 %
30 %	7.93	+67 %
50 %	4.75	reference
70 %	3.39	−29 %
80 %	2.97	−37 %
100 %	2.38	−50 %

TCO vs energy price

Energy price (€/kWh)	Energy opex/year	€/GPU-hour (70 % util.)	Change vs base
0.034 (solar PPA)	1,604 €	3.00	−12 %
0.116 (industrial ES, base)	5,475 €	3.39	reference
0.160 (European average)	7,550 €	3.54	+4 %
0.200 (high tariff)	9,437 €	3.67	+8 %

Energy has a moderate impact on total TCO (8–12 % variation between the extremes), because hardware capex dominates. Over a very long amortisation (5 years) with a solar PPA, however, energy drops from 6 % to 1 % of total TCO and the differential is amplified. The price of energy matters more for the carbon footprint (CSRD) than for TCO when capex is dominant.

TCO vs PUE

PUE	Cooling overhead	Energy/year (0.116 €/kWh)	€/GPU-hour (70 % util.)
1.15 (liquid cooling, new facilities)	+15 %	2,166 €	3.21
1.20 (liquid, modern datacenter)	+20 %	2,259 €	3.23
1.48 (facilities <5 years, Uptime 2025)	+48 %	3,490 €	3.33
1.54 (global average, Uptime 2025)	+54 %	3,627 €	3.39
1.80 (legacy colocation)	+80 %	4,260 €	3.47

The difference between PUE 1.15 (liquid) and 1.80 (legacy) is barely ~8 % of the €/GPU-hour at 70 % utilisation, because energy is only a fraction of TCO. PUE matters far more for the absolute energy cost and for CSRD reporting than for total TCO when hardware is the dominant component.

TCO vs amortisation years

Amortisation	Capex/year (average node, USD)	€/GPU-hour (70 % util., base opex scenario)
3 years	59,500 USD (~55,300 €)	3.39
4 years	44,625 USD (~41,500 €)	2.99
5 years	35,700 USD (~33,200 €)	2.72

Stretching the amortisation from 3 to 5 years lowers the €/GPU-hour by ~20 %, assuming the hardware remains competitive and the resale market supports the residual value. With the refresh cycle accelerated by Blackwell GB200/GB300, a 5-year amortisation carries a higher risk of technological obsolescence.

Sensitivity heat map (€/GPU-hour at 70 % utilisation, base scenario)

	PUE 1.15	PUE 1.54	PUE 1.80
3-yr amort., solar PPA (0.034 €)	2.72	2.74	2.76
3-yr amort., industrial (0.116 €)	3.21	3.39	3.47
5-yr amort., industrial (0.116 €)	2.54	2.72	2.80
3-yr amort., high tariff (0.200 €)	3.44	3.67	3.78

Decision table: cost/control/sovereignty Pareto

The table below crosses the four dimensions with no implicit hierarchy; the ordinal reading depends on each organisation’s constraints.

Option	€/GPU-hour	Initial capex	Full stack control	EU sovereignty	Elasticity	Operational risk
On-prem (util. >70 %, low opex)	2.40–3.00	high (150–207k USD/node)	total	total	none	hardware failure, idle
On-prem (util. <50 %, base opex)	4.75–7.93	high	total	total	none	capex with no return
Neocloud 3-year reserved (CoreWeave, Lambda)	1.49–2.10 USD	none	partial (API)	depends on the provider	rigid contract	minimal interruption
Neocloud on-demand (Lambda, Spheron)	2.49–3.44 USD	none	partial	depends	total	no interruption
AWS p5 on-demand	6.88 USD	none	minimal	NO (CLOUD Act)	total	no interruption
AWS p5 3-year reserved	~2.97 USD	financial commitment	minimal	NO (CLOUD Act)	rigid	no interruption
Sovereign EU cloud (Scaleway, Nebius EU)	2.15–3.85 USD	none	partial	yes (EU)	total	no interruption
Hybrid on-prem base + EU cloud peak	2.00–3.50 (weighted)	medium	high	yes (EU)	elastic peak	operational complexity

“EU sovereignty” column: US hyperscalers (AWS, GCP, Azure) are subject to the US CLOUD Act regardless of the datacenter region. Nebius has a Dutch legal entity; CoreWeave is a US company. See the full analysis in sovereign on-premise vs hyperscalers.

“Full stack control” column: on-prem lets you choose the driver version, the kernel, the NCCL configuration, MIG partitioning, and any system parameter. Cloud options offer control at container/pod level, with the hypervisor and firmware opaque.

The cost/sovereignty Pareto frontier for GDPR data excludes US hyperscalers, leaving: on-prem, sovereign EU cloud, and the hybrid. Among those three, the deciding variable is sustained utilisation and the predictability of traffic (see capacity planning for on-premise LLM inference).

Integration with the series FinOps model

The all-in €/GPU-hour of on-prem is the number that feeds the cost allocation pipeline of the series:

Cost-per-token identity (cost per token and per request): engine throughput × €/GPU-hour → €/1M tokens.
Chargeback and showback (chargeback and showback in GPU multitenancy): the all-in €/GPU-hour is the internal price charged to each tenant of the multi-tenant cluster.
Utilisation as a lever (GPU utilisation as FinOps): raising utilisation from 50 % to 80 % cuts the €/GPU-hour by 37 % without changing the hardware, the highest ROI in on-prem FinOps.
Capacity planning (capacity planning for on-premise LLM inference): the number of nodes to buy depends on the percentile of base load you want to cover in iron.
Cloud comparison (cloud GPU: on-demand, reserved and spot): the all-in €/GPU-hour goes head to head with the cloud price in table A7 to compute the break-even.

Sources

Spheron · LLM Inference On-Premise vs GPU Cloud: 2026 Cost and Break-Even Analysis (abr. 2026) — https://www.spheron.network/blog/llm-inference-on-premise-vs-cloud/
Introl · GPU Infrastructure TCO Model: 5-Year Cost Analysis for Enterprise AI (abr. 2026) — https://introl.com/blog/gpu-infrastructure-tco-5-year-cost-model
GMI Cloud · NVIDIA H100 GPU Pricing: 2026 Rent vs. Buy Cost Analysis (abr. 2026) — https://www.gmicloud.ai/en/blog/nvidia-h100-gpu-pricing-2026-rent-vs-buy-cost-analysis
Uptime Institute · Global Data Center Survey 2025 (PDF oficial) — https://datacenter.uptimeinstitute.com/rs/711-RIA-145/images/2025.Annual.Survey.Report.pdf
Uptime Institute · Global Data Center PUE Stalls at 1.54 (comunicado, oct. 2025) — https://mgrid.org/2025/10/01/uptime-institute-data-center-pue-stagnation-2025-liquid-cooling/
GlobalPetrolPrices · Spain Business Electricity Price (sept. 2025) — https://www.globalpetrolprices.com/Spain/electricity_prices/
PV Tech · European Solar PPA Prices Fall Below 35 €/MWh in Q3 2025 (oct. 2025) — https://www.pv-tech.org/european-solar-ppa-prices-fall-below-35-mwh-q3-2025/
Xi Computers · Supermicro SYS-821GE-TNHR 8U GPU Server Pricing (jun. 2026) — https://www.xicomputer.com/Solutions/Data-Center-Servers/SYS-821GE-TNHR.asp
Encoradvisors · Data Center Colocation Pricing 2026 — https://encoradvisors.com/data-center-colocation-pricing/
CloudZero · H100 GPU Cost In 2026: Buy, Rent, and Cloud Pricing Compared — https://www.cloudzero.com/blog/h100-gpu-cost/
Spheron · AI Inference Power Consumption and GPU Electricity Costs: 2026 Guide — https://www.spheron.network/blog/ai-inference-power-electricity-cost-2026/
NVIDIA · DGX SuperPOD H100 Electrical Specifications (documentación oficial) — https://docs.nvidia.com/dgx-superpod/design-guides/dgx-superpod-data-center-design-h100/latest/electrical.html

Sovereign on-premise vs hyperscalers: the case with data (cost, energy, performance and sovereignty)

Sun, 14 Jun 2026 05:30:00 +0200

Notation: amounts in euros (N €), decimals with a point; when a source quotes dollars it is marked “USD”. Data centred on Europe and Spain. The dollar symbol is not used (formula delimiter).

What this article covers

Synthesis article of the series (S2), and the heart of the proposal: the case with data for sovereign on-premise against the hyperscalers and the European cloud. Up to here, each track measured its own axis, cost per token (FinOps), goodput (benchmarking), energy and carbon; here the four are crossed, with a fourth dimension no US technical comparison puts up front: data sovereignty. The aim is to answer, with numbers and not with ideology, the question that underpins any investment in an AI platform: serve on your own iron, on a European cloud or on a hyperscaler? And to do it honestly: on-prem does not always win, and saying when it wins and when it does not is what makes the recommendation credible.

The framework: four axes, not one number

The mistake in almost every comparison is reducing the decision to the hourly cost of a GPU. The real decision crosses four axes, and only by seeing them together do you decide well:

Axis	Question	Who measures it in the series
Cost (TCO)	how much does it cost to serve, all in?	FinOps (A2–A8)
Performance	does it meet the SLO, at what goodput?	Benchmarking (B2–B8)
Energy and carbon	how many watts and grams per token?	Energy (C2–C8)
Sovereignty	under which jurisdiction does the data live?	GDPR / EU AI Act / CSRD

The first three are quantifiable and meet in the cost per token; the fourth is a constraint that can rule out an option however cheap it is. The synthesis consists of scoring each option on all four and deciding on the Pareto frontier, not on whichever axis happens to suit.

The cost axis: TCO and break-even

The real on-premise cost is amortised capex plus opex, and its cost per effective hour depends on utilisation:

$$\text{effective cost/GPU-hour (on-prem)} = \frac{\text{annual amortised capex} + \text{annual opex}}{8760 \times \text{utilisation}}$$

This formula is the key to the whole debate: the cost per useful hour of on-prem rises as utilisation falls, because capex is paid whether the GPU is working or idle. The 2026 data:

Data point	Value	Source
On-prem cost 8×H100 (floor, high util.)	~2.83 USD/GPU-hour all-in	Spheron
Annual on-prem cost (floor)	~237,000 USD/year	Spheron
AWS H100 (p5.48xlarge)	4.10–6.88 USD/GPU-hour	Spheron
AWS 8-GPU on-demand annual (100 % util)	287,000–482,000 USD/year	Spheron
Sovereign European cloud (Lyceum/Scaleway)	from 2–2.73 €/GPU-hour, zero-egress	Lyceum, Scaleway

The break-even against AWS on-demand falls around 50–83 % utilisation depending on region and tariff; below ~70 % utilisation, the cloud wins on TCO, and above it, on-prem (Spheron). For very high-utilisation workloads, on-prem pays for itself in less than 4 months (Lenovo).

Three-year TCO: the full calculation for an 8×H100 node

Abstract numbers do not convince a committee; a three-year model with declared line items does. Take a sovereign 8×H100 node in Spain and compare it, for the same work, with AWS and with a European cloud. Declared assumptions: amortisation over 3 years, energy at a solar PPA of 32.5 €/MWh (with grid backup), PUE 1.3, and two utilisation scenarios (50 % and 80 %).

On-premise (own node), annual line items:

Line item	Annual value	Note
Amortised capex (node ~270,000 € / 3 years)	~90,000 €	8×H100 server + network + storage
Energy (≈10.4 kW × PUE 1.3 × 8760 h)	~3,850 € (at 32.5 €/MWh)	with solar PPA; at grid tariff, ~12–18 k €
Operation, cooling, maintenance	~25,000 €	prorated staff, support, spares
Datacenter space (rack, connectivity)	~12,000 €	colocation or own datacenter
Annual total	~131,000 €	independent of utilisation

At a fixed 131,000 €/year, the cost per token depends only on how many tokens you generate, that is, on utilisation:

Utilisation	Useful GPU-hours/year	Cost/1M tokens
50 %	~35,000	~2.9 €
65 %	~45,500	~2.2 €
80 %	~56,000	~1.8 € (with cheap grid, ~1.1 €)

Sovereign European cloud (Scaleway/Lyceum), pay per use: at ~2.2 €/GPU-hour with zero-egress, the cost per token is constant with utilisation (you only pay for what you use): ~1.5–2.2 €/1M tokens depending on model and batching, with no capex and no idle risk.

Hyperscaler (AWS p5), on-demand: at 4.10–6.88 USD/GPU-hour (≈3.8–6.4 €), the cost per token works out at ~2–3.5 €/1M tokens, and on top of that you have to add the egress, quite apart from the fact that for GDPR data the sovereignty axis rules it out already.

The reading of the model is the thesis of the whole of S2: at 50 % utilisation, on-prem (~2.9 €) does not beat the European cloud (~1.8 €); at 80 % with cheap energy (~1.1 €), it beats it comfortably. The crossover sits, as the literature says, around 65–70 %. Investing in on-prem is, fundamentally, a bet that you will sustain high utilisation, and that bet is won with scheduling, not with hardware.

The uncomfortable reality: the utilisation almost nobody reaches

Here is the honest figure missing from the “on-prem is always cheaper” speeches: most inference teams in production operate at 40–65 % GPU utilisation, because of traffic variability and the limits of batching; the 80–90 % assumption that makes on-prem attractive is rarely reached outside batch-only pipelines (Spheron).

This changes the naive conclusion: if your real utilisation is 50 %, on-prem is not cheaper than the cloud, because the capex you pay for the idle GPU eats you. That is why utilisation is not a detail, it is the variable that decides the cost axis, and it connects directly with the FinOps track (the idle of A2, the chargeback of A5) and with scheduling: raising utilisation is what makes on-prem pay. A badly used cluster of your own is more expensive than the cloud; a well scheduled one is far cheaper. The cost question is not “on-prem or cloud?”, it is “can I sustain high utilisation?”.

The hidden costs of the cloud: egress

The cloud has its own small print: egress costs (taking data out of the provider). In the hyperscalers, moving data out or between regions is billed, and in AI workloads with a lot of data movement (datasets, checkpoints, embeddings) it can be a significant line item that does not appear in the GPU-hour price. The advantage of the sovereign European cloud: most of them (Lyceum among others) have adopted the zero-egress model, they do not charge for moving data out or between regions (Lyceum). When comparing, the real hyperscaler cost is GPU-hour + egress + other charges, not just the GPU-hour; ignoring it artificially inflates the hyperscaler’s competitiveness.

An example of the order of magnitude: a platform moving 50 TB/month outbound (datasets, checkpoints, responses served to systems outside the provider) at a typical egress tariff of ~0.08–0.09 €/GB pays ~4,000–4,500 €/month, that is ~50,000 €/year on egress alone, a line item the size of a third of the cost of a node of your own, invisible in the GPU-hour price. On the European cloud with zero-egress that line item is zero; on-prem, internal traffic is not billed either. That is why a fair comparison must model egress according to the real data pattern: for workloads with a lot of outbound movement, it can invert the ranking between hyperscaler and European cloud. The cloud bill is not the GPU-hour; it is the GPU-hour plus everything you move.

To this is added contract and lock-in risk: the hyperscaler’s on-demand GPU tariffs can change, commitment discounts (reserved/savings plans) tie you for 1–3 years, and migrating out, because of egress and coupling to proprietary services, has a real exit cost. On-prem and the European cloud with standard APIs (Kubernetes, S3 compatible) reduce that coupling: the same manifest and the same vLLM run on your cluster or on Scaleway without rewriting. Operational sovereignty, being able to move the workload without rebuilding it, is a value that does not appear in the tariff but weighs on a three-year decision.

The performance axis: the provider does not decide, goodput does

One point that simplifies the synthesis: performance does not depend on the provider, it depends on the hardware and the configuration. An H100 gives the same goodput on your cluster, on Scaleway or on AWS, served with the same vLLM and the same config. What decides performance is the goodput under your SLO (track B), not who hosts the GPU. So, in a comparison at equal hardware, the performance axis neutralises itself: what changes between options is cost, energy and sovereignty. The exception: if a provider gives you access to newer hardware (B200, GB200) ahead of your on-prem buying cycle, then the cloud can win on performance per GPU, a real argument in favour of the cloud for staying at the hardware frontier without capex.

The energy axis: the European and Spanish advantage

Here on-prem (or cloud) in Spain or France has a structural advantage over a hyperscaler in a dirty region. Recalling the data from the energy track:

Location	Grid carbon (gCO₂/kWh)	Price (indicative)
France (nuclear)	~20–60	low and stable
Spain (renewables + gas)	~150–170	low, volatile; solar PPA ~32.5 €/MWh
Germany	~363	high
Hyperscaler (region depends on provider)	depends; often not selectable	provider tariff

The same workload in France emits ~9× less carbon per token than in Germany, and in Spain, with a solar PPA at 32.5 €/MWh (an all-time low), the electricity cost, 30–50 % of the TCO, is low and, under contract, predictable. A sovereign cluster in Spain or France controls where the energy is consumed and with what carbon; a hyperscaler gives you the region it gives you, often with no choice of grid intensity. For CSRD reporting, that selectability is a quantifiable advantage of on-prem and the European cloud.

In concrete numbers: the example 8×H100 node (~10.4 kW × PUE 1.3 ≈ 118,000 kWh/year) emits, depending on the grid, ~2.4 t CO₂/year in France (~20 gCO₂/kWh) against ~43 t CO₂/year in Germany (~363 gCO₂/kWh), the same machine, the same work, ~18× the difference in reportable footprint just by choosing the location. That decision, which a hyperscaler in an imposed region does not let you take, is exactly what on-prem and the sovereign European cloud put in your hands. The energy axis is not an environmental detail: it is cost (the price of the kWh), compliance (CSRD) and sovereignty (control of location) all at once.

The sovereignty axis: the one that does not depend on utilisation

And here is the axis that invalidates the cheapest option if the data is sensitive. US hyperscalers are subject to the US CLOUD Act: US authorities can demand data held by a US company even if it sits in a European datacenter. For data subject to the GDPR, that is a compliance risk. Sovereign European clouds operate under EU/EFTA jurisdiction, providing data residency and GDPR compliance, and are exempt from the US CLOUD Act (Lyceum · sovereign providers). On-prem of your own is the maximum degree of sovereignty: the data does not leave your cluster.

The key difference from the other axes: sovereignty does not depend on utilisation or on volume. However much a hyperscaler cheapens the GPU-hour, for GDPR data it is not an option, the jurisdiction risk is not offset by price. It links with the ENS × ISO 42001 × EU AI Act controls and the EU AI Act mapping: compliance is a hard constraint, not an axis to optimise.

The four instruments that turn sovereignty into a concrete constraint rather than a slogan:

Instrument	What it requires	Implication for the architecture
US CLOUD Act	gives the US access to data held by US companies, wherever it is	a US hyperscaler does not guarantee jurisdictional residency even if the datacenter is in the EU
GDPR	residency and processing of personal data under EU law	requires an EU/EFTA provider or your own iron for personal data
EU AI Act	traceability, risk management and records for AI systems	favours the full control of the stack (logs, datasets, models) that on-prem gives
CSRD	verifiable reporting of environmental footprint	the selectable energy (clean grid, PPA) of on-prem and the European cloud is auditable

The operational conclusion: for a European entity processing personal data or deploying high-risk AI, three of the four technical axes can favour the hyperscaler and it can still lose, because the fourth axis, sovereignty, acts as a prior filter. That is why S2 orders the decision like this: first the sovereignty filter (which rules out the hyperscaler for GDPR data), then the optimisation of cost, performance and energy among the options that pass the filter (sovereign on-prem and European cloud).

The scorecard: the three options scored

Crossing the four axes for the three realistic options of a European platform (order-of-magnitude, illustrative figures):

Option	Cost/1M tok	Break-even	Energy/carbon	Sovereignty
Sovereign on-prem (ES/FR)	~1.1 € (high util.) / ~3 € (low)	>65–70 % util.	controllable (clean grid, PPA)	total (EU)
Sovereign European cloud	~1.5–2.2 €	no capex, pay per use	EU, zero-egress	high (EU)
Hyperscaler (US)	~2–3.5 € + egress	no capex	imposed region	not EU (CLOUD Act)

The reading of the scorecard: for data subject to the GDPR, the US hyperscaler is ruled out by the sovereignty axis, however competitive its tariff. The real decision comes down to sovereign on-prem vs sovereign European cloud, and there it is decided by utilisation and volume.

When each option wins

The honest recommendation, by scenario:

Scenario	Winning option	Why
High, sustained volume (util. >65–70 %), GDPR data	Sovereign on-prem	lowest cost/token + total sovereignty
Variable or growing volume, GDPR data	Sovereign European cloud	sovereignty without capex/idle risk
Low volume or sporadic peak	European cloud (per use)	you do not amortise the capex
No sovereignty requirement, hardware frontier	Hyperscaler	access to new hardware without capex
Hybrid (base + peak)	On-prem + European cloud (burst)	cheap base of your own, sovereign elastic peak

On-prem makes sense when there is very high, predictable utilisation (80 %+), strict sovereignty requirements, or a hyperscaler contract that works out expensive (Spheron). For sovereign platforms with a sustained base load, the winning pattern is usually the hybrid: sovereign on-prem for the high-utilisation base (where the cost per token is unbeatable) and sovereign European cloud for the peak and for growth (elastic, no capex, keeping EU jurisdiction). The best of both without giving up sovereignty.

Sizing the hybrid: how much on iron, how much on cloud

The hybrid is not “a bit of each”; it is sized with one figure: the base load percentile. The rule is to put on on-prem the load that is almost always present (the load that keeps the GPU at 75–85 %) and to send to the European cloud only the peaks that, if covered with iron, would leave GPUs idle most of the time. An example with a realistic traffic profile:

Load band	% of the time	Where to serve	Why
Base (p0–p70)	always	on-prem (1 node 8×H100 at ~80 %)	minimum cost/token, high util. guaranteed
Middle (p70–p95)	daily peak hours	on-prem if it fits, otherwise European cloud	elasticity without idle capex
Peak (p95–p100)	sporadic	sovereign European cloud (burst)	absurd to buy iron for a rare peak

With this split, the base amortises your own node at high utilisation (~1.1–1.8 €/1M tokens) and the peak is paid per use with no idle penalty (~1.5–2.2 €/1M tokens), all under EU jurisdiction. The expensive mistake is the opposite: sizing the on-prem for the peak, in which case the GPU spends most of its time idle at 30–40 %, the cost per token shoots up above 3 € and the cloud would have been cheaper. You size the iron for the base, not for the peak; the peak is exactly what the cloud does well. This principle connects with the capacity planning and the scheduling (Kueue/Volcano) of the series: the hybrid only works if the scheduler fills your own node before overflowing to the cloud.

Assumptions and sensitivity

The whole comparison hangs on assumptions that have to be declared, because moving them moves the conclusion:

Assumption	If it rises	Effect
Utilisation	50 % → 80 %	on-prem goes from losing to clearly winning
Energy price	expensive region → France/PPA	lowers on-prem TCO and carbon
Amortisation period	24 → 36 months	lowers the on-prem cost per hour
Volume	< 2M tok/day → much more	crosses the break-even towards on-prem
Egress (hyperscaler)	low → high	makes the hyperscaler dearer than the European cloud

The rule: no on-prem vs cloud comparison is valid without fixing these assumptions. One that says “on-prem is 3× cheaper” without declaring the assumed utilisation is propaganda; one that fixes utilisation, energy price, period and volume is a data point. The dossier must present the case with explicit assumptions and a sensitivity analysis, which is what makes it defensible before a committee that questions them.

Decision checklist

To take S2 from theory to decision, the questions that order the choice, in order:

Is the data subject to the GDPR, or is the system high-risk under the EU AI Act? If so, the US hyperscaler is ruled out on sovereignty; you choose between on-prem and European cloud. If not, the hyperscaler enters the cost comparison.
Can I sustain utilisation above 65–70 % on the base load? If so, on-prem wins on cost for that base. If not, the European cloud avoids paying capex for idle GPUs.
Does the traffic profile have marked peaks? If so, hybrid: base on iron, peak on European cloud. Size the iron for the base, never for the peak.
How much data do I take out of the provider each month? Model the egress; with a lot of movement, the zero-egress of the European cloud, or on-prem, win clearly.
Which electricity grid and at what price? France or Spain with a PPA lower TCO and carbon; include it in the model and in the CSRD report.
Have I fixed utilisation, energy, period and volume in writing? Without those four declared assumptions, the number is not defensible.

Whoever answers these six questions with data, not with intuition, has the case built. The series' recommendation for a sovereign European platform with a sustained base load is stable: sovereign on-prem for the high-utilisation base plus sovereign European cloud for the peak, with the hyperscaler reserved only for workloads with no sovereignty requirement where frontier hardware is needed without capex.

Limits and traps (data-driven)

Unrealistic assumed utilisation. The 80–90 % that makes on-prem win rarely happens in production (40–65 % typical). Model your real utilisation, not the ideal one.
Comparing only the GPU-hour. TCO includes energy, operation, cooling, egress (cloud) and capex (on-prem). Compare totals with the same assumptions.
Ignoring sovereignty. For GDPR data, the sovereignty axis rules out the hyperscaler before cost does; it is not negotiable with price.
Forgetting the hybrid. It is not “all on-prem or all cloud”; the base-plus-peak pattern usually dominates.
Data in USD. US comparisons are in dollars and with dirty regions; convert them to euros and to your region’s grid (Spain/France) for your case.

The synthesis of S2, in one sentence: for sovereign European data, the decision is not on-prem vs cloud in the abstract, but sovereign on-prem (high utilisation) plus sovereign European cloud (peak) against a hyperscaler that the sovereignty axis rules out, and utilisation is the variable that splits the base between the first two. The rest of the series gives the numbers for each axis; this one crosses them into the recommendation. The next synthesis article (S3) sizes the investment; this one decides the architecture.

Sources

Spheron · LLM Inference On-Premise vs GPU Cloud: 2026 Cost and Break-Even — https://www.spheron.network/blog/llm-inference-on-premise-vs-cloud/
Lenovo Press · On-Premise vs Cloud: Generative AI TCO (2026) — https://lenovopress.lenovo.com/lp2368-on-premise-vs-cloud-generative-ai-total-cost-of-ownership-2026-edition
Lyceum · EU Sovereign Inference Platform Comparison (2026) — https://lyceum.technology/magazine/eu-sovereign-inference-platform-comparison/
Lyceum · Sovereign Cloud Providers 2026 — https://lyceum.technology/magazine/sovereign-cloud-providers-2026/
Scaleway · H100 GPU instance (precio €, soberanía UE) — https://www.scaleway.com/en/h100/
Nerd Level Tech · GPU Cloud TCO 2026: hidden fees, egress costs — https://nerdleveltech.com/gpu-cloud-comparison-2026-the-real-cost-of-ai-compute

Cost, performance and energy: the three axes of LLM inference and the identity that ties them together

Sat, 13 Jun 2026 03:30:00 +0200

Notation: amounts in euros (N €), decimals with a point. Cost and energy references are European (France, Germany, Spain), because the aim is to defend a sovereign European architecture; when a source quotes prices in dollars it is flagged as “USD”. The dollar symbol is not used (on this site it is a formula delimiter).

What this series is

This is the first instalment of a series of 28 data articles (not opinion pieces) about the three dimensions that decide whether a sovereign European LLM platform is sustainable: cost (FinOps), performance (benchmarking) and energy. The aim is to have a body of verifiable figures, with their methodology, to support architecture decisions with numbers rather than intuition. This first article sets the frame: what each axis measures, how they link to one another, an end-to-end worked example with European data and the 2026 references that will be used as a base.

The quantitative thesis, in one sentence: the three axes are not independent. Throughput (tokens per second) is the common denominator of cost per token and energy per token, and energy is, on top of that, between 30 % and 50 % of the cost itself. Optimising one moves the other two. And in Europe, where the cluster sits (France, Germany, Spain) changes both the electricity cost and the carbon per token at once.

The three axes and their metrics

Axis	Question	Key metrics	Unit	Tool family
Cost (FinOps)	How much does each unit of work cost?	cost/1M tokens (CPM), cost/request, €/GPU-hour, utilisation	€/1M tok, €/h	OpenCost, Kubecost, CloudZero
Performance	How much work per second and at what latency?	TTFT, TPOT/ITL, throughput, goodput, P50/P99	tok/s, ms	GuideLLM, AIPerf, LLMPerf, MLPerf Inference
Energy	How many watts per unit of work?	energy/token, average power, PUE, carbon	J/tok, Wh, kWh, gCO₂	Zeus, CodeCarbon, MLPerf Power, Kepler

Each axis has its own in-depth introductory article in this same batch; here only how they fit together matters. It is worth breaking each one down a little more before crossing them.

Cost axis in detail

Metric	Definition	Why it matters
CPM (cost/1M tokens)	cluster cost ÷ tokens produced	the comparable unit on-prem vs cloud
Cost per request	cost attributed to one complete request	billing by real usage
€/GPU-hour	hourly cost of a GPU (amortised or rented)	the basis of any calculation
Utilisation	fraction of the GPU actually used	the lever that moves everything else
Fixed vs variable cost	amortised capex vs energy/operations	defines the break-even point

Performance axis in detail

Metric	Definition	Dominant phase
TTFT	latency to the first token	prefill
TPOT / ITL	time between output tokens	decode
Throughput	tokens or requests per second of the system	both
Goodput	throughput that meets the SLO (not the raw figure)	the honest number
P95 / P99	latency percentiles (not the mean)	the real experience

Energy axis in detail

Metric	Definition	Unit
Average power	average instantaneous consumption	W
Energy per token	energy per generated token	J/token
Total energy	integral of power over time	Wh, kWh
PUE	datacenter overhead (cooling, losses)	ratio ≥ 1
Carbon	energy × grid intensity	gCO₂eq

The identity that ties the three axes together: throughput

The cost per million tokens (CPM) is the cluster’s cost divided by what it produces:

$$\text{CPM} = \frac{\text{cluster cost per hour}}{\text{throughput (tok/s)} \times 3600 / 10^6}$$

Energy per token has exactly the same form, with power in the numerator:

$$\text{energy per token (J)} = \frac{\text{average power (W)}}{\text{throughput (tok/s)}}$$

The two share the same denominator: throughput. Hence the key operational consequence: every performance improvement that raises tokens/s lowers both cost per token and energy per token at the same time. These are not three separate optimisations; throughput moves them together. That is why performance benchmarking (axis 2) is, at bottom, also a FinOps tool and an energy efficiency tool.

The CPM↔throughput relationship is direct: the higher the throughput, the lower the CPM. It is the identity on which the whole cost analysis of the series is built.

Worked example: unit economics of an 8×H100 node in Europe

To see the identity in action, an end-to-end calculation on an example node (8×H100 SXM, Llama 3.1 70B FP16 served with vLLM), with European prices. Order-of-magnitude figures; the methodology is what matters.

Starting data:

Parameter	Value	Source
European cloud rental (Scaleway, per H100)	2.73 €/h	Scaleway
→ 8×H100 node on-demand	~21.84 €/h	8 × 2.73
Amortised on-prem cost (estimate)	~10–13 €/h	capex 24–36 months + opex
Throughput	2,800 tok/s	reference benchmark
Power per H100 (TDP)	up to 700 W	NVIDIA
Reference PUE	1.4	typical of a datacenter

Cost per million tokens. At 2,800 tok/s, hourly throughput is 2,800 × 3,600 / 10⁶ = 10.08 million tokens/hour. Therefore:

$$\text{CPM (cloud)} = \frac{21{.}84}{10{.}08} \approx 2{.}17 \text{ €/1M tok}$$

With the amortised on-prem cost (~11 €/h) it drops to ~1.09 €/1M tok, the difference between renting from a provider and amortising your own iron.

Energy per token. The power of the 8 cards is 8 × 700 = 5,600 W (board only). Over 2,800 tok/s:

$$\text{energy per token} = \frac{5,600}{2,800} = 2 \text{ J/token (board)}$$

Over a million tokens that is 2 × 10⁶ J = 0.56 kWh (GPU only). Applying PUE 1.4: ~0.78 kWh per million tokens. At French wholesale prices (~0.058 €/kWh) that is ~0.045 € of electricity per million tokens; in Germany (~0.091 €/kWh), ~0.071 €, some 57 % more, because of the grid, not the model.

The link. If a serving optimisation raised throughput from 2,800 to 4,200 tok/s (+50 %), the CPM would fall to ~1.45 €/1M tok and energy per token to ~1.33 J, both at once, through the same denominator. That is the identity, with numbers.

Energy is inside the cost

The second link: energy is not an axis separate from cost, it is a large component of it. 2026 reference data:

Figure	Value	Source
Electricity as a share of TCO (5 years)	30–40 % (>50 % at high density 24/7)	Spheron
Day-ahead wholesale France (Mar-2026)	~58 €/MWh (0.058 €/kWh)	TradingEconomics
Day-ahead wholesale Spain	~54 €/MWh (0.054 €/kWh)	TradingEconomics
Day-ahead wholesale Germany	~91 €/MWh (0.091 €/kWh)	TradingEconomics
Electricity cost/token at 80 % util	¼ of that at 20 % util	Spheron

A methodological warning: the wholesale price (day-ahead) is not what a company pays; industrial retail adds grid charges and taxes (roughly ×1.5–2). But the relative ordering holds: France and Spain, cheap; Germany, expensive.

The third link, utilisation, closes the triangle: power is consumed even when the GPU is idle, so spreading the fixed cost (energy included) over more tokens (more throughput, more occupancy) lowers all three axes at once. The electricity cost per token at 80 % utilisation is a quarter of what it is at 20 %.

Sensitivity analysis: what moves the cost per token

Four levers dominate the CPM, and it is worth knowing their elasticity before defending any figure:

Lever	Change	Effect on CPM	Mechanism
Throughput	+50 % tok/s	−33 % CPM	direct denominator
Utilisation	20 % → 80 %	up to −75 % on the electricity portion	spreads fixed cost
Location (energy)	Germany → France	lowers the electricity portion (~37 % less €/kWh)	wholesale FR < DE
Amortise vs rent	21.84 → 11 €/h	−50 % CPM	removes the provider’s margin
Quantisation (FP8/INT4)	+throughput, −VRAM/KV	lowers CPM and J/token	same identity

The reading: the CPM is not a number, it is a function of utilisation, throughput, location (energy price) and ownership model. Any on-prem vs cloud comparison that does not fix those assumptions is not comparable, and that is exactly the job of the synthesis article.

Simplified TCO model (on-premise)

The real on-premise cost is not the GPU alone: it is amortised capex + opex, over a hardware cycle of 24–36 months. Reference structure of an inference node:

Component	Type	Typical weight	Notes
GPUs	capex	the bulk	amortise over 24–36 months
Server, network, BMC	capex	medium	NICs, switching, out-of-band management
Electricity	opex	30–50 % of TCO	depending on country (FR/ES cheap, DE expensive) and PUE
Cooling	opex	inside the PUE	PUE 1.2–1.6 typical
Maintenance and support	opex	medium	spares, contracts, RMA
Staff and operations	opex	variable	the hidden cost that cloud rental includes

The difference between cloud rental (21.84 €/h in the example) and the real amortised cost (~10–13 €/h) is this breakdown: the provider charges a margin for absorbing capex, cooling and operations. The break-even point appears when sustained volume exceeds ~2M tokens/day, with typical amortisation in 6–12 months (Spheron). Below that threshold, renting usually wins; above it, amortising. Article A8 develops the full model and synthesis S3 connects it with sizing.

Reference: the price of a GPU-hour in European sovereign cloud

The other side of the on-prem comparison is the price of renting the GPU. For a sovereign European platform, what matters is not AWS/GCP/Azure (subject to the US CLOUD Act), but the European providers:

Provider	H100 (€/hour)	Location	Sovereignty
Scaleway	from 2.73	Paris, Warsaw	EU jurisdiction, exempt from US CLOUD Act
OVHcloud	+40–45 % vs Scaleway	France, Germany	EU jurisdiction, exempt from US CLOUD Act
Hyperscalers (reference)	3–7 USD	global	subject to US CLOUD Act

The data: Scaleway offers the H100 from 2.73 €/h in European datacenters (Paris, Warsaw), with models and data under EU jurisdiction, immune to extraterritorial regulation; OVHcloud serves it from France and Germany, ~40–45 % more expensive than Scaleway as of March 2026. Both are exempt from the US CLOUD Act, the foundational sovereignty requirement for data subject to GDPR (Scaleway, DeployBase).

The implication for the proposal: the comparison is not only cost, it is cost + sovereignty. An on-prem cluster of your own or a European cloud keep the data under EU jurisdiction; a US hyperscaler does not, however cheap the GPU-hour comes out. That is the leg that the cost axis on its own does not capture, and that synthesis S2 quantifies.

Energy and carbon reference: France, Germany, Spain

Here the choice of European country changes both the electricity cost and the carbon per token. It is the most important table for a sovereign proposal:

Country	Wholesale (€/MWh, Mar-2026)	Grid carbon (gCO₂/kWh)	Profile
France	~58	~20–60	nuclear: cheap and very clean
Spain	~54	~150–170	renewables + gas: cheap, medium carbon
Germany	~91	~363 (2024 average)	coal/gas + renewables: expensive and dirty

Sources: day-ahead wholesale prices (TradingEconomics), German carbon ~363 gCO₂/kWh (2024 average, Statista via Wikipedia), nuclear ~17–35 gCO₂e/kWh (EEA), Spain ~167 gCO₂/kWh (2020). The PUE multiplies all of that energy: at 1.4, for every kWh of GPU the datacenter spends 1.4 kWh with cooling included.

The calculation, on the example node (0.78 kWh per million tokens, with PUE): in France (~40 gCO₂/kWh) it is ~31 gCO₂ per million tokens; in Germany (~363 gCO₂/kWh), ~283 gCO₂ per million, a factor of ~9×. That is, the same load served in France emits of the order of 9× less carbon per token than in Germany, and costs ~37 % less in electricity, without touching a single line of the stack, purely because of the country’s grid. Siting the cluster in France or Spain is at once a cost lever, a carbon lever and a compliance lever. Hourly grid intensity is obtained from ElectricityMaps, the reference source for these calculations.

Scale context: global datacenters consumed ~460 TWh in 2025 (≈1.8 % of the world’s electricity), and the IEA projects passing 800 TWh in 2028, which is why energy efficiency per token stops being a detail and becomes a design constraint, and why carbon enters the European regulatory equation.

Sovereignty and regulation: the axis that does not appear in the CPM

For a European platform there is a fourth factor that the three numerical axes do not capture, and that can invalidate the lowest cost figure: jurisdiction. Three fronts that turn “the cheapest GPU-hour” into “the cheapest GPU-hour that complies”:

Front	What it requires	Architectural implication
GDPR + US CLOUD Act	personal data under a jurisdiction accessible to US authorities is exposed	European cloud (Scaleway/OVHcloud) or on-prem keep the data in the EU
EU AI Act	obligations for high-risk systems (traceability, control, documentation)	on-prem control makes auditability and compliance easier
CSRD / emissions reporting	reporting the carbon footprint (scope 2)	a low-carbon grid (France) improves the reported figure

The key point for the proposal: a US hyperscaler can offer a competitive GPU-hour, but it puts the data under the US CLOUD Act, which for data subject to GDPR is a compliance risk, not a saving. That is why this series’ comparison is not made in USD against US providers: it is made in euros, against options that keep data sovereignty, on-prem of your own or European cloud. Cost, energy and carbon are measurable axes; sovereignty is the constraint that decides which options even enter the table. This links with the mapping of the EU AI Act onto the on-premise architecture and with the technical controls ENS × ISO 42001 × EU AI Act.

The operational conclusion: France and Spain not only come out cheaper and cleaner than Germany; being EU jurisdiction, they meet the sovereignty constraint that a US hyperscaler does not meet at any price. Three advantages that reinforce each other, and that the proposal can defend with the same table.

Methodology: how to read (and not fool yourself with) these figures

Five warnings that apply to all the data in this series:

Board vs node. The GPU’s power (700 W per H100) is the floor; the complete node (CPU, NICs, power supplies, fans) consumes more. Always compare the same boundary.
Wholesale vs retail. The day-ahead price is not what the company pays; industrial retail adds grid charges and taxes. Use whichever matches your contract.
Measured vs estimated. A wattmeter figure and an nvidia-smi estimate are not the same thing; the difference is quantified in the energy articles.
Amortisation assumptions. The on-prem cost/hour depends on the hardware cycle (24 vs 36 months) and on the assumed utilisation; changing those assumptions moves the CPM more than any serving optimisation.
Comparability. Two CPM or J/token figures are only comparable if model, precision, context length, load and country match. Most public comparisons (and in USD) do not fix those assumptions, which is why the series reproduces the ones that matter with its own harness and European data.

The combined scorecard

The destination of the series is a single board where each model/configuration/location is scored on all the axes at once, to decide on a Pareto frontier rather than on a single number. An illustrative example (order-of-magnitude figures) on the same 8×H100 node:

Candidate	CPM (€/1M)	Goodput (tok/s)	Energy (J/tok)	Carbon (gCO₂/1M)	Sovereignty
A · 70B FP16, on-prem France	~1.09	~2,800	~2.0	low (grid ~40 gCO₂/kWh)	EU
B · 70B FP8, on-prem France	~0.75	~4,000	~1.4	lower	EU
C · Scaleway on-demand	~2.17	~2,800	n/a	low (FR)	EU
D · US hyperscaler	~3.0	~2,800	n/a	depends on region	not EU

The reading: B dominates A on the three intrinsic axes thanks to FP8. C (European cloud) is more expensive per token but shifts capex and operations, while keeping sovereignty. D (hyperscaler) may be competitive on cost but loses sovereignty, an axis that for GDPR data is not negotiable. No isolated number decides: the choice comes from seeing the whole row.

The trap the board avoids: optimising a single axis. Each cell will be filled in by an article in the series with measured data.

Where each axis is measured: from design to production

Each axis is measured at three different moments, with different tools and increasing precision. Confusing them is a common source of figures that do not add up:

Axis	Design (estimate)	Benchmark (controlled)	Production (continuous)
Cost	datasheet, GPU-hour price	CPM measured over synthetic load	OpenCost/Kubecost per team and token
Performance	theoretical throughput (roofline)	GuideLLM/AIPerf with SLO	OTel/Langfuse over real traffic
Energy	TDP × hours	Zeus/MLPerf Power on the bench	Kepler/DCGM per pod in the cluster

The rule: a design figure serves for sizing, but not for defending a proposal; for that you need a reproducible benchmark figure, and to operate, a continuous production one. When someone disputes a number in the proposal, the answer is not “a blog says so”, but “here is the bench, here is the methodology, reproduce it”.

Glossary of units and conversions

Quantity	Unit	Conversion / reference
Energy	joule (J), watt-hour (Wh)	1 Wh = 3,600 J; 1 kWh = 3.6 MJ = 3,600,000 J
Power	watt (W)	1 W = 1 J/s; H100 ≈ 700 W TDP
Energy per token	J/token	= average power (W) ÷ throughput (tok/s)
Throughput	tok/s	tokens generated per second by the system
Cost per token	€/1M tok (CPM)	= cost/hour ÷ (tok/s × 3,600 / 10⁶)
Latency	ms	TTFT (first token), TPOT/ITL (between tokens)
PUE	ratio (≥1)	total DC energy ÷ compute energy
Carbon	gCO₂eq	= energy (kWh) × PUE × grid intensity (gCO₂/kWh)

A useful conversion to keep in your head: at a constant 700 W, an H100 consumes ~0.7 kWh per hour, ~16.8 kWh a day and ~6,130 kWh a year (board only, no PUE). Multiplied by the PUE and your country’s energy price (FR ~0.058, ES ~0.054, DE ~0.091 €/kWh wholesale), that number is the basis of the electricity portion of the TCO.

How the rest of the series fits together

The 28 articles group into four tracks; this is the map:

Track	What it contributes	Closes with
FinOps (8)	measuring and allocating cost, TCO model	cost/token comparable on-prem vs European cloud
LLM benchmarking (8)	measuring performance reproducibly	Pareto frontier of engines
Energy (8)	measuring watts and carbon per token	energy in the TCO and in European regulation
Synthesis (4)	crossing the three axes	scorecard, on-prem vs cloud, sizing, harness

The three articles that follow this one in the batch introduce each axis in depth: their frameworks and the state of the art. From there, each article adds a layer of data to the final scorecard.

The thread running through all 28 instalments is the one in this article: cost, performance and energy are a single problem with three faces, tied together by throughput and by the energy that lives inside the cost, and in Europe, by the country where the cluster sits. Anyone who optimises one face while ignoring the others will end up with a pretty number and a platform that does not hold up. The work of the series is to produce the figures, measured, dated, European and reproducible, that allow a sovereign architecture to be defended before any committee with the only answer that closes a technical debate: the data and the bench to reproduce it.

Sources

Scaleway · H100 GPU instance (€/h price, EU sovereignty) — https://www.scaleway.com/en/h100/
DeployBase · OVHcloud GPU Pricing (European sovereignty) — https://deploybase.ai/articles/ovhcloud-gpu-pricing
TradingEconomics · electricity price France / Germany / Spain — https://tradingeconomics.com/france/electricity-price
EEA · emission intensity of electricity in Europe — https://www.eea.europa.eu/en/analysis/indicators/greenhouse-gas-emission-intensity-of-1
Electricity sector in Germany (carbon ~363 gCO₂/kWh) — https://en.wikipedia.org/wiki/Electricity_sector_in_Germany
Spheron · AI Inference Power & Cost Economics (2026) — https://www.spheron.network/blog/ai-inference-cost-economics-2026/
NVIDIA · H100 (TDP/specs) — https://www.nvidia.com/en-us/data-center/h100/

Technical figure	How it is obtained	Use in TCO	Use in regulatory reporting
\(P_{\text{IT}}\) (W)	DCGM / NVML / IPMI	Energy cost formula	\(E_{\text{IT,annual}}\) for PUE, WUE, ERF, REF
Throughput (tok/s)	Engine metrics (vLLM, TGI)	€/token, Wh/token	Energy intensity of inference
PUE (measured)	Meters on PDUs and IT equipment (ISO/IEC 30134-2)	Multiplier in the TCO formula	Mandatory ReportENER KPI
Annual \(I_{\text{grid}}\) (gCO₂/kWh)	REE / MITECO / national factors	gCO₂/token for carbon-aware	Emission factor for location-based Scope 2 ESRS E1-6
Hourly \(I_{\text{grid}}\)	ElectricityMaps / Carbon Aware SDK	Carbon-aware scheduling	Location-based Scope 2 with hourly granularity (best practice)
\(W_{\text{in}}\) (m³)	Datacenter water meters	Cost of water (where applicable)	WUE for ReportENER
\(E_{\text{reuse}}\) (kWh)	Heat export meters	Possible waste heat revenue	ERF for ReportENER
\(E_{\text{res,tot}}\) (kWh)	Energy bills with a renewable attribute / GOs	Energy cost with a PPA	REF for ReportENER; market-based Scope 2 for ESRS E1-6
Hardware embodied carbon (gCO₂eq)	Supplier LCA (AWS methodology, where available)	—	Scope 3 cat. 1 for ESRS E1-6

Tco on lo0 — Tech Blog

Energy in TCO and in regulation: cost, PUE, CSRD and the new European reporting obligation for datacenters

TL;DR

Track context

Energy as a TCO component

The reference model

The annual energy cost formula

Sensitivity to energy price and to PUE

Energy as a fraction of total TCO

European regulatory framework 2024-2026

Map of the rules

Energy Efficiency Directive: EED 2023/1791

Delegated Regulation (EU) 2024/1364

Phase 2: rating scheme and sustainability label

CSRD and ESRS E1: corporate emissions reporting

Spain: PNIEC context and the Royal Decree in progress

Regulatory metrics: definitions and formulas

PUE — Power Usage Effectiveness

WUE — Water Usage Effectiveness

ERF — Energy Reuse Factor

REF — Renewable Energy Factor

CUE — Carbon Usage Effectiveness

Master table: rule × what it requires × metric/threshold × date

How Wh/token and gCO₂/token connect with the required reporting

The conversion chain

From Wh/token to the regulatory KPI: aggregation

From Wh/token to gCO₂/token as a CSRD accounting figure

Worked example: from J/token to tCO₂eq/year

The 500 kW threshold and on-premise AI datacenters

Which datacenters does it affect in practice?

PUE as an operational figure and as a regulatory figure

Sensitivity of gCO₂/token to PUE and energy price

Connection between operator metrics and regulator obligations

The map of required data

The minimum measurement system for compliance

Minimum Performance Standards: the 2027-2028 outlook

See also

Sources

Full TCO of an on-premise GPU cluster: from capex to the all-in €/GPU-hour and the break-even against cloud

TL;DR

The model: declared assumptions

Capex breakdown per 4×H100 SXM5 node

GPUs

Server / HGX baseboard

InfiniBand NDR network

NVMe storage

Rack, PDU and datacenter connectivity

Capex summary per 4×H100 SXM5 node

Opex breakdown per 4×H100 SXM5 node (annual)

Energy

Staff / operations

Maintenance and support

Depreciation (for accounting purposes)

Annual opex summary per 4×H100 SXM5 node (base scenario, 8-node cluster)

Deriving the all-in €/GPU-hour

Formula

€/GPU-hour table by utilisation and scenario

From €/GPU-hour to €/1M tokens

Break-even on-prem vs cloud

The break-even formula

Break-even table by cloud mode and on-prem scenario

Reading the break-even table

Sensitivity analysis

TCO vs utilisation

TCO vs energy price

TCO vs PUE

TCO vs amortisation years

Sensitivity heat map (€/GPU-hour at 70 % utilisation, base scenario)

Decision table: cost/control/sovereignty Pareto

Integration with the series FinOps model

Sources

Sovereign on-premise vs hyperscalers: the case with data (cost, energy, performance and sovereignty)

What this article covers

The framework: four axes, not one number

The cost axis: TCO and break-even

Three-year TCO: the full calculation for an 8×H100 node

The uncomfortable reality: the utilisation almost nobody reaches

The hidden costs of the cloud: egress

The performance axis: the provider does not decide, goodput does

The energy axis: the European and Spanish advantage