Cost, performance and energy: the three axes of LLM inference and the identity that ties them together
Contents
Notation: amounts in euros (N €), decimals with a point. Cost and energy references are European (France, Germany, Spain), because the aim is to defend a sovereign European architecture; when a source quotes prices in dollars it is flagged as “USD”. The dollar symbol is not used (on this site it is a formula delimiter).
What this series is
This is the first instalment of a series of 28 data articles (not opinion pieces) about the three dimensions that decide whether a sovereign European LLM platform is sustainable: cost (FinOps), performance (benchmarking) and energy. The aim is to have a body of verifiable figures, with their methodology, to support architecture decisions with numbers rather than intuition. This first article sets the frame: what each axis measures, how they link to one another, an end-to-end worked example with European data and the 2026 references that will be used as a base.
The quantitative thesis, in one sentence: the three axes are not independent. Throughput (tokens per second) is the common denominator of cost per token and energy per token, and energy is, on top of that, between 30 % and 50 % of the cost itself. Optimising one moves the other two. And in Europe, where the cluster sits (France, Germany, Spain) changes both the electricity cost and the carbon per token at once.
The three axes and their metrics
| Axis | Question | Key metrics | Unit | Tool family |
|---|---|---|---|---|
| Cost (FinOps) | How much does each unit of work cost? | cost/1M tokens (CPM), cost/request, €/GPU-hour, utilisation | €/1M tok, €/h | OpenCost, Kubecost, CloudZero |
| Performance | How much work per second and at what latency? | TTFT, TPOT/ITL, throughput, goodput, P50/P99 | tok/s, ms | GuideLLM, AIPerf, LLMPerf, MLPerf Inference |
| Energy | How many watts per unit of work? | energy/token, average power, PUE, carbon | J/tok, Wh, kWh, gCO₂ | Zeus, CodeCarbon, MLPerf Power, Kepler |
Each axis has its own in-depth introductory article in this same batch; here only how they fit together matters. It is worth breaking each one down a little more before crossing them.
Cost axis in detail
| Metric | Definition | Why it matters |
|---|---|---|
| CPM (cost/1M tokens) | cluster cost ÷ tokens produced | the comparable unit on-prem vs cloud |
| Cost per request | cost attributed to one complete request | billing by real usage |
| €/GPU-hour | hourly cost of a GPU (amortised or rented) | the basis of any calculation |
| Utilisation | fraction of the GPU actually used | the lever that moves everything else |
| Fixed vs variable cost | amortised capex vs energy/operations | defines the break-even point |
Performance axis in detail
| Metric | Definition | Dominant phase |
|---|---|---|
| TTFT | latency to the first token | prefill |
| TPOT / ITL | time between output tokens | decode |
| Throughput | tokens or requests per second of the system | both |
| Goodput | throughput that meets the SLO (not the raw figure) | the honest number |
| P95 / P99 | latency percentiles (not the mean) | the real experience |
Energy axis in detail
| Metric | Definition | Unit |
|---|---|---|
| Average power | average instantaneous consumption | W |
| Energy per token | energy per generated token | J/token |
| Total energy | integral of power over time | Wh, kWh |
| PUE | datacenter overhead (cooling, losses) | ratio ≥ 1 |
| Carbon | energy × grid intensity | gCO₂eq |
The identity that ties the three axes together: throughput
The cost per million tokens (CPM) is the cluster’s cost divided by what it produces:
$$\text{CPM} = \frac{\text{cluster cost per hour}}{\text{throughput (tok/s)} \times 3600 / 10^6}$$Energy per token has exactly the same form, with power in the numerator:
$$\text{energy per token (J)} = \frac{\text{average power (W)}}{\text{throughput (tok/s)}}$$The two share the same denominator: throughput. Hence the key operational consequence: every performance improvement that raises tokens/s lowers both cost per token and energy per token at the same time. These are not three separate optimisations; throughput moves them together. That is why performance benchmarking (axis 2) is, at bottom, also a FinOps tool and an energy efficiency tool.
The CPM↔throughput relationship is direct: the higher the throughput, the lower the CPM. It is the identity on which the whole cost analysis of the series is built.
Worked example: unit economics of an 8×H100 node in Europe
To see the identity in action, an end-to-end calculation on an example node (8×H100 SXM, Llama 3.1 70B FP16 served with vLLM), with European prices. Order-of-magnitude figures; the methodology is what matters.
Starting data:
| Parameter | Value | Source |
|---|---|---|
| European cloud rental (Scaleway, per H100) | 2.73 €/h | Scaleway |
| → 8×H100 node on-demand | ~21.84 €/h | 8 × 2.73 |
| Amortised on-prem cost (estimate) | ~10–13 €/h | capex 24–36 months + opex |
| Throughput | 2,800 tok/s | reference benchmark |
| Power per H100 (TDP) | up to 700 W | NVIDIA |
| Reference PUE | 1.4 | typical of a datacenter |
Cost per million tokens. At 2,800 tok/s, hourly throughput is 2,800 × 3,600 / 10⁶ = 10.08 million tokens/hour. Therefore:
$$\text{CPM (cloud)} = \frac{21{.}84}{10{.}08} \approx 2{.}17 \text{ €/1M tok}$$With the amortised on-prem cost (~11 €/h) it drops to ~1.09 €/1M tok, the difference between renting from a provider and amortising your own iron.
Energy per token. The power of the 8 cards is 8 × 700 = 5,600 W (board only). Over 2,800 tok/s:
$$\text{energy per token} = \frac{5,600}{2,800} = 2 \text{ J/token (board)}$$Over a million tokens that is 2 × 10⁶ J = 0.56 kWh (GPU only). Applying PUE 1.4: ~0.78 kWh per million tokens. At French wholesale prices (~0.058 €/kWh) that is ~0.045 € of electricity per million tokens; in Germany (~0.091 €/kWh), ~0.071 €, some 57 % more, because of the grid, not the model.
The link. If a serving optimisation raised throughput from 2,800 to 4,200 tok/s (+50 %), the CPM would fall to ~1.45 €/1M tok and energy per token to ~1.33 J, both at once, through the same denominator. That is the identity, with numbers.
Energy is inside the cost
The second link: energy is not an axis separate from cost, it is a large component of it. 2026 reference data:
| Figure | Value | Source |
|---|---|---|
| Electricity as a share of TCO (5 years) | 30–40 % (>50 % at high density 24/7) | Spheron |
| Day-ahead wholesale France (Mar-2026) | ~58 €/MWh (0.058 €/kWh) | TradingEconomics |
| Day-ahead wholesale Spain | ~54 €/MWh (0.054 €/kWh) | TradingEconomics |
| Day-ahead wholesale Germany | ~91 €/MWh (0.091 €/kWh) | TradingEconomics |
| Electricity cost/token at 80 % util | ¼ of that at 20 % util | Spheron |
A methodological warning: the wholesale price (day-ahead) is not what a company pays; industrial retail adds grid charges and taxes (roughly ×1.5–2). But the relative ordering holds: France and Spain, cheap; Germany, expensive.
The third link, utilisation, closes the triangle: power is consumed even when the GPU is idle, so spreading the fixed cost (energy included) over more tokens (more throughput, more occupancy) lowers all three axes at once. The electricity cost per token at 80 % utilisation is a quarter of what it is at 20 %.
Sensitivity analysis: what moves the cost per token
Four levers dominate the CPM, and it is worth knowing their elasticity before defending any figure:
| Lever | Change | Effect on CPM | Mechanism |
|---|---|---|---|
| Throughput | +50 % tok/s | −33 % CPM | direct denominator |
| Utilisation | 20 % → 80 % | up to −75 % on the electricity portion | spreads fixed cost |
| Location (energy) | Germany → France | lowers the electricity portion (~37 % less €/kWh) | wholesale FR < DE |
| Amortise vs rent | 21.84 → 11 €/h | −50 % CPM | removes the provider’s margin |
| Quantisation (FP8/INT4) | +throughput, −VRAM/KV | lowers CPM and J/token | same identity |
The reading: the CPM is not a number, it is a function of utilisation, throughput, location (energy price) and ownership model. Any on-prem vs cloud comparison that does not fix those assumptions is not comparable, and that is exactly the job of the synthesis article.
Simplified TCO model (on-premise)
The real on-premise cost is not the GPU alone: it is amortised capex + opex, over a hardware cycle of 24–36 months. Reference structure of an inference node:
| Component | Type | Typical weight | Notes |
|---|---|---|---|
| GPUs | capex | the bulk | amortise over 24–36 months |
| Server, network, BMC | capex | medium | NICs, switching, out-of-band management |
| Electricity | opex | 30–50 % of TCO | depending on country (FR/ES cheap, DE expensive) and PUE |
| Cooling | opex | inside the PUE | PUE 1.2–1.6 typical |
| Maintenance and support | opex | medium | spares, contracts, RMA |
| Staff and operations | opex | variable | the hidden cost that cloud rental includes |
The difference between cloud rental (21.84 €/h in the example) and the real amortised cost (~10–13 €/h) is this breakdown: the provider charges a margin for absorbing capex, cooling and operations. The break-even point appears when sustained volume exceeds ~2M tokens/day, with typical amortisation in 6–12 months (Spheron). Below that threshold, renting usually wins; above it, amortising. Article A8 develops the full model and synthesis S3 connects it with sizing.
Reference: the price of a GPU-hour in European sovereign cloud
The other side of the on-prem comparison is the price of renting the GPU. For a sovereign European platform, what matters is not AWS/GCP/Azure (subject to the US CLOUD Act), but the European providers:
| Provider | H100 (€/hour) | Location | Sovereignty |
|---|---|---|---|
| Scaleway | from 2.73 | Paris, Warsaw | EU jurisdiction, exempt from US CLOUD Act |
| OVHcloud | +40–45 % vs Scaleway | France, Germany | EU jurisdiction, exempt from US CLOUD Act |
| Hyperscalers (reference) | 3–7 USD | global | subject to US CLOUD Act |
The data: Scaleway offers the H100 from 2.73 €/h in European datacenters (Paris, Warsaw), with models and data under EU jurisdiction, immune to extraterritorial regulation; OVHcloud serves it from France and Germany, ~40–45 % more expensive than Scaleway as of March 2026. Both are exempt from the US CLOUD Act, the foundational sovereignty requirement for data subject to GDPR (Scaleway, DeployBase).
The implication for the proposal: the comparison is not only cost, it is cost + sovereignty. An on-prem cluster of your own or a European cloud keep the data under EU jurisdiction; a US hyperscaler does not, however cheap the GPU-hour comes out. That is the leg that the cost axis on its own does not capture, and that synthesis S2 quantifies.
Energy and carbon reference: France, Germany, Spain
Here the choice of European country changes both the electricity cost and the carbon per token. It is the most important table for a sovereign proposal:
| Country | Wholesale (€/MWh, Mar-2026) | Grid carbon (gCO₂/kWh) | Profile |
|---|---|---|---|
| France | ~58 | ~20–60 | nuclear: cheap and very clean |
| Spain | ~54 | ~150–170 | renewables + gas: cheap, medium carbon |
| Germany | ~91 | ~363 (2024 average) | coal/gas + renewables: expensive and dirty |
Sources: day-ahead wholesale prices (TradingEconomics), German carbon ~363 gCO₂/kWh (2024 average, Statista via Wikipedia), nuclear ~17–35 gCO₂e/kWh (EEA), Spain ~167 gCO₂/kWh (2020). The PUE multiplies all of that energy: at 1.4, for every kWh of GPU the datacenter spends 1.4 kWh with cooling included.
The calculation, on the example node (0.78 kWh per million tokens, with PUE): in France (~40 gCO₂/kWh) it is ~31 gCO₂ per million tokens; in Germany (~363 gCO₂/kWh), ~283 gCO₂ per million, a factor of ~9×. That is, the same load served in France emits of the order of 9× less carbon per token than in Germany, and costs ~37 % less in electricity, without touching a single line of the stack, purely because of the country’s grid. Siting the cluster in France or Spain is at once a cost lever, a carbon lever and a compliance lever. Hourly grid intensity is obtained from ElectricityMaps, the reference source for these calculations.
Scale context: global datacenters consumed ~460 TWh in 2025 (≈1.8 % of the world’s electricity), and the IEA projects passing 800 TWh in 2028, which is why energy efficiency per token stops being a detail and becomes a design constraint, and why carbon enters the European regulatory equation.
Sovereignty and regulation: the axis that does not appear in the CPM
For a European platform there is a fourth factor that the three numerical axes do not capture, and that can invalidate the lowest cost figure: jurisdiction. Three fronts that turn “the cheapest GPU-hour” into “the cheapest GPU-hour that complies”:
| Front | What it requires | Architectural implication |
|---|---|---|
| GDPR + US CLOUD Act | personal data under a jurisdiction accessible to US authorities is exposed | European cloud (Scaleway/OVHcloud) or on-prem keep the data in the EU |
| EU AI Act | obligations for high-risk systems (traceability, control, documentation) | on-prem control makes auditability and compliance easier |
| CSRD / emissions reporting | reporting the carbon footprint (scope 2) | a low-carbon grid (France) improves the reported figure |
The key point for the proposal: a US hyperscaler can offer a competitive GPU-hour, but it puts the data under the US CLOUD Act, which for data subject to GDPR is a compliance risk, not a saving. That is why this series’ comparison is not made in USD against US providers: it is made in euros, against options that keep data sovereignty, on-prem of your own or European cloud. Cost, energy and carbon are measurable axes; sovereignty is the constraint that decides which options even enter the table. This links with the mapping of the EU AI Act onto the on-premise architecture and with the technical controls ENS × ISO 42001 × EU AI Act.
The operational conclusion: France and Spain not only come out cheaper and cleaner than Germany; being EU jurisdiction, they meet the sovereignty constraint that a US hyperscaler does not meet at any price. Three advantages that reinforce each other, and that the proposal can defend with the same table.
Methodology: how to read (and not fool yourself with) these figures
Five warnings that apply to all the data in this series:
- Board vs node. The GPU’s power (700 W per H100) is the floor; the complete node (CPU, NICs, power supplies, fans) consumes more. Always compare the same boundary.
- Wholesale vs retail. The day-ahead price is not what the company pays; industrial retail adds grid charges and taxes. Use whichever matches your contract.
- Measured vs estimated. A wattmeter figure and an
nvidia-smiestimate are not the same thing; the difference is quantified in the energy articles. - Amortisation assumptions. The on-prem cost/hour depends on the hardware cycle (24 vs 36 months) and on the assumed utilisation; changing those assumptions moves the CPM more than any serving optimisation.
- Comparability. Two CPM or J/token figures are only comparable if model, precision, context length, load and country match. Most public comparisons (and in USD) do not fix those assumptions, which is why the series reproduces the ones that matter with its own harness and European data.
The combined scorecard
The destination of the series is a single board where each model/configuration/location is scored on all the axes at once, to decide on a Pareto frontier rather than on a single number. An illustrative example (order-of-magnitude figures) on the same 8×H100 node:
| Candidate | CPM (€/1M) | Goodput (tok/s) | Energy (J/tok) | Carbon (gCO₂/1M) | Sovereignty |
|---|---|---|---|---|---|
| A · 70B FP16, on-prem France | ~1.09 | ~2,800 | ~2.0 | low (grid ~40 gCO₂/kWh) | EU |
| B · 70B FP8, on-prem France | ~0.75 | ~4,000 | ~1.4 | lower | EU |
| C · Scaleway on-demand | ~2.17 | ~2,800 | n/a | low (FR) | EU |
| D · US hyperscaler | ~3.0 | ~2,800 | n/a | depends on region | not EU |
The reading: B dominates A on the three intrinsic axes thanks to FP8. C (European cloud) is more expensive per token but shifts capex and operations, while keeping sovereignty. D (hyperscaler) may be competitive on cost but loses sovereignty, an axis that for GDPR data is not negotiable. No isolated number decides: the choice comes from seeing the whole row.
The trap the board avoids: optimising a single axis. Each cell will be filled in by an article in the series with measured data.
Where each axis is measured: from design to production
Each axis is measured at three different moments, with different tools and increasing precision. Confusing them is a common source of figures that do not add up:
| Axis | Design (estimate) | Benchmark (controlled) | Production (continuous) |
|---|---|---|---|
| Cost | datasheet, GPU-hour price | CPM measured over synthetic load | OpenCost/Kubecost per team and token |
| Performance | theoretical throughput (roofline) | GuideLLM/AIPerf with SLO | OTel/Langfuse over real traffic |
| Energy | TDP × hours | Zeus/MLPerf Power on the bench | Kepler/DCGM per pod in the cluster |
The rule: a design figure serves for sizing, but not for defending a proposal; for that you need a reproducible benchmark figure, and to operate, a continuous production one. When someone disputes a number in the proposal, the answer is not “a blog says so”, but “here is the bench, here is the methodology, reproduce it”.
Glossary of units and conversions
| Quantity | Unit | Conversion / reference |
|---|---|---|
| Energy | joule (J), watt-hour (Wh) | 1 Wh = 3,600 J; 1 kWh = 3.6 MJ = 3,600,000 J |
| Power | watt (W) | 1 W = 1 J/s; H100 ≈ 700 W TDP |
| Energy per token | J/token | = average power (W) ÷ throughput (tok/s) |
| Throughput | tok/s | tokens generated per second by the system |
| Cost per token | €/1M tok (CPM) | = cost/hour ÷ (tok/s × 3,600 / 10⁶) |
| Latency | ms | TTFT (first token), TPOT/ITL (between tokens) |
| PUE | ratio (≥1) | total DC energy ÷ compute energy |
| Carbon | gCO₂eq | = energy (kWh) × PUE × grid intensity (gCO₂/kWh) |
A useful conversion to keep in your head: at a constant 700 W, an H100 consumes ~0.7 kWh per hour, ~16.8 kWh a day and ~6,130 kWh a year (board only, no PUE). Multiplied by the PUE and your country’s energy price (FR ~0.058, ES ~0.054, DE ~0.091 €/kWh wholesale), that number is the basis of the electricity portion of the TCO.
How the rest of the series fits together
The 28 articles group into four tracks; this is the map:
| Track | What it contributes | Closes with |
|---|---|---|
| FinOps (8) | measuring and allocating cost, TCO model | cost/token comparable on-prem vs European cloud |
| LLM benchmarking (8) | measuring performance reproducibly | Pareto frontier of engines |
| Energy (8) | measuring watts and carbon per token | energy in the TCO and in European regulation |
| Synthesis (4) | crossing the three axes | scorecard, on-prem vs cloud, sizing, harness |
The three articles that follow this one in the batch introduce each axis in depth: their frameworks and the state of the art. From there, each article adds a layer of data to the final scorecard.
The thread running through all 28 instalments is the one in this article: cost, performance and energy are a single problem with three faces, tied together by throughput and by the energy that lives inside the cost, and in Europe, by the country where the cluster sits. Anyone who optimises one face while ignoring the others will end up with a pretty number and a platform that does not hold up. The work of the series is to produce the figures, measured, dated, European and reproducible, that allow a sovereign architecture to be defended before any committee with the only answer that closes a technical debate: the data and the bench to reproduce it.
See also
- Comparison of serving engines (vLLM/SGLang/TRT-LLM/Dynamo) — the synthesis of the performance axis: which engine wins at each point of the goodput-latency frontier with measured data.
- LLM quality benchmarks: MMLU, MT-Bench, HELM and the leaderboards that matter — the quality dimension that complements performance: how public leaderboards let you calibrate the right model for each SLO.
- LLM energy efficiency leaderboards — the synthesis of the energy axis: the J/token rankings and how to read them without being fooled by the benchmark’s setup.
- Cloud GPU: comparison of prices, commitment and sovereign neoclouds — the cost axis piece for cloud or hybrid architectures: on-demand, spot and reserved prices from European providers in 2026.
- TCO of the on-premise GPU cluster: amortisation, energy and infrastructure — the cost axis piece for your own architectures: the real €/GPU-hour with CAPEX, energy and operations broken down.
- Measurement bias and reproducibility in LLM benchmarks — the methodological traps that make two benchmarks of the same engine give incompatible results: the basis for trusting this series’ data.
- From the watt to carbon: PUE, grid intensity and the real cost of a token — how to convert the GPU’s watts into gCO₂eq and into electricity euros, closing the loop between the energy axis and the cost axis.
- Energy efficiency levers in LLM inference — the optimisations that move all three axes at once: quantisation, batching and engine as levers that reduce J/token, €/token and improve goodput simultaneously.
- The reproducible harness for measuring cost, performance and energy — the test bench that makes this article’s methodology concrete: how to produce the data for the three axes with a single reproducible experiment.
Sources
- Scaleway · H100 GPU instance (€/h price, EU sovereignty) — https://www.scaleway.com/en/h100/
- DeployBase · OVHcloud GPU Pricing (European sovereignty) — https://deploybase.ai/articles/ovhcloud-gpu-pricing
- TradingEconomics · electricity price France / Germany / Spain — https://tradingeconomics.com/france/electricity-price
- EEA · emission intensity of electricity in Europe — https://www.eea.europa.eu/en/analysis/indicators/greenhouse-gas-emission-intensity-of-1
- Electricity sector in Germany (carbon ~363 gCO₂/kWh) — https://en.wikipedia.org/wiki/Electricity_sector_in_Germany
- Spheron · AI Inference Power & Cost Economics (2026) — https://www.spheron.network/blog/ai-inference-cost-economics-2026/
- NVIDIA · H100 (TDP/specs) — https://www.nvidia.com/en-us/data-center/h100/