GPU chargeback and showback in multi-tenancy: how to split cluster cost between teams

Mon, 15 Jun 2026 04:00:00 +0200

Notation: amounts in euros (N €), decimals with a point. The dollar sign is not used (on this site it is a formula delimiter).

TL;DR

A cluster of 4×H100 SXM at ~1.40 €/GPU-hour (depreciated capex plus energy) shared between three teams with different occupancy produces a monthly chargeback report of 3 rows and an additional idle row that nobody claimed. Without tooling, that idle cost vanishes, diluted into the total. With OpenCost + LiteLLM + Kueue, attribution operates on three orthogonal planes: the iron (OpenCost, €/GPU-hour per namespace/label), token consumption (LiteLLM, €/token per key/team/model) and the scheduler quota (Kueue, GPUs reserved per ClusterQueue). Crossing the three produces the number that goes to finance: team B consumed X million tokens at Y €/1M tok, the GPU cost it Z €, and it still has 2 GPUs borrowed from the cohort that will cost it W € if it holds on to them next month.

Showback vs chargeback: the FinOps Foundation definition

The distinction is not one of technology but of accounting formality (FinOps Foundation — Invoicing & Chargeback, FinOps Foundation — Data Analysis and Showback):

Concept	Definition	Moves money	Requires
Showback	visibility of consumption and its cost per team; the report arrives, the budget does not change	No	metrics + attribution
Chargeback	the cost is transferred to the team’s or product’s P&L as real expenditure	Yes	showback + accounting policy + financial system

Two points from the framework worth pinning down:

Showback is a requirement of any FinOps practice; chargeback is optional and depends on the organisation’s accounting policy supporting transfers between cost centres.
Neither is “more mature” than the other. The narrative that chargeback is the “grown-up” version is false according to the framework itself. The natural sequence is: showback → trust in the data → chargeback if policy allows it.

When to use each

Situation	Recommended mode
First attribution cycle; teams do not trust the data yet	Showback
Teams with their own budget in a financial system	Chargeback
Shared cluster with no P&L separation per team	Showback with visible idle cost
Teams with a committed GPU availability SLA	Chargeback (real reservation)
Research/experimentation workloads with no formal budget	Showback + threshold alert

GPU cost attribution with OpenCost

Allocation API: attribution parameters

The OpenCost /allocation API is the piece that turns Prometheus metrics into a cost report per dimension (OpenCost — API):

Parameter	Values	Use in multi-tenancy
`window`	`today`, `7d`, `lastmonth`, RFC3339 range	window of the monthly report
`aggregate`	`namespace`, `label:LABEL`, `annotation:NAME`, `pod`, `controller`	attribution dimension
`includeIdle`	`true` / `false`	adds an `__idle__` row to the report
`shareIdle`	`true` / `false`	distributes idle among the non-idle allocations (proportional to non-idle cost)
`idleByNode`	`true` / `false`	computes idle per node instead of per cluster
`filter`	`namespace:"llm-prod"`, `label:equipo:"datos"`	restrict to one specific team

Query for a monthly report per team, with idle visible as a separate row:

curl -G http://localhost:9003/allocation \
 -d window=lastmonth \
 -d aggregate=label:equipo \
 -d includeIdle=true \
 -d shareIdle=false \
 -d resolution=10m

With shareIdle=false (the default), OpenCost returns idle as a separate __idle__ entry, which makes visible how much was paid for unused capacity. With shareIdle=true, that idle is distributed among the tenants in proportion to their non-idle cost: each absorbs its share of the waste, which is the right behaviour for a chargeback that penalises whoever is responsible for the idle.

Labelling inference pods

For aggregate=label:equipo to work, the inference pods need the right label. Two methods:

Label in the pod spec (the most direct):

# vLLM Deployment for the datos team
metadata:
 labels:
 equipo: datos
 producto: rag-prod
 modelo: llama-70b
 entorno: prod

Annotation (when the label is already taken by another convention):

# aggregate as an annotation
aggregate=annotation:finops.io/equipo

OpenCost supports aggregate=label:KEY and aggregate=annotation:KEY with the same syntax; the choice between the two depends on the cluster’s labelling convention.

Shared costs and idle: the three modes

API mode	Behaviour	When
`includeIdle=false` (default)	Ignores idle; total cost looks lower than it is	Never in production
`includeIdle=true, shareIdle=false`	Idle in a separate `__idle__` row	Showback: visibility of waste
`includeIdle=true, shareIdle=true`	Idle distributed among tenants proportionally	Chargeback: the team pays its share of idle
`shareIdle=true, idleByNode=true`	Idle distributed per node (finer-grained if there are dedicated nodes)	Chargeback with heterogeneous nodes

For infrastructure with nodes dedicated per team, idleByNode=true is fairer: the idle of a node dedicated to team A is not transferred to team B.

LiteLLM as the attribution point for token consumption

Virtual keys and teams

LiteLLM materialises token chargeback with four mechanisms (LiteLLM — Virtual Keys, LiteLLM — Setting Team Budgets, LiteLLM — Spend Tracking):

Mechanism	What it does
Virtual key with `max_budget`	monthly budget per key; the key is blocked once it is exhausted
`budget_duration`	reset window: `30d`, `7d`, `24h`; the proxy runs a daily cron that resets according to the duration
`team_id`	groups keys; spend accumulates in `LiteLLM_TeamTable` per team
`tags`	tag each request; they allow budgets per tag (cost centre)
Spend logs	one row per request with `team_id`, `model`, `prompt_tokens`, `completion_tokens`, `response_cost`

Configuring teams with a monthly budget

# litellm-config.yaml — per-team budgets with the declared on-prem cost
model_list:
 - model_name: llama-3-70b-onprem
 litellm_params:
 model: openai/llama-3-70b
 api_base: http://vllm-svc:8000/v1
 input_cost_per_token: 0.00000140 # 1.40 €/1M tok (€/GPU-hour ÷ throughput)
 output_cost_per_token: 0.00000140
 - model_name: llama-3-8b-onprem
 litellm_params:
 model: openai/llama-3-8b
 api_base: http://vllm-8b-svc:8000/v1
 input_cost_per_token: 0.00000035 # 0.35 €/1M tok (1 GPU, higher throughput)
 output_cost_per_token: 0.00000035

litellm_settings:
 success_callback: ["langfuse"] # or any external logger

# Teams (via the /team/new API or config)
# POST /team/new
# {
# "team_alias": "equipo-datos",
# "max_budget": 800, # 800 € per month
# "budget_duration": "30d",
# "tpm_limit": 500000, # max tokens/min
# "rpm_limit": 1000 # max requests/min
# }

The input_cost_per_token key comes from the identity in article A4: the €/GPU-hour from OpenCost divided by the benchmark throughput. With that value, the response_cost of each request reflects the real cost of the on-prem iron for that model.

Endpoints for querying spend per team

# Team spend accumulated over the period
GET /team/info?team_id=equipo-datos

# Team spend per model (aggregated spend_logs table)
GET /global/spend/logs?team_id=equipo-datos&start_date=2026-06-01&end_date=2026-06-30

# Total spend across all teams
GET /global/spend/teams

The response includes spend (accumulated cost in the configured currency), total_tokens, prompt_tokens, completion_tokens and a per-model breakdown, the direct fields for the monthly chargeback report.

Joining €/GPU-hour (iron) with tokens per team

The two attribution planes, OpenCost (iron) and LiteLLM (tokens), are crossed with one key: the model_name and the namespace. The join happens outside the tools (in a BI pipeline or a SQL query over the LiteLLM database and the data exported from OpenCost):

coste_por_token_real = OpenCost.GPU_cost_namespace / LiteLLM.total_tokens_team

When the on-prem cost is properly declared in input_cost_per_token, LiteLLM already does this calculation internally and the response_cost is correct. The explicit join serves to validate: if the sum of LiteLLM’s response_cost per team does not match the GPU cost OpenCost assigns to the namespace, there is drift in input_cost_per_token that has to be corrected (changed throughput, a new optimisation, more replicas).

Kueue as the GPU budget/quota mechanism

ClusterQueue, LocalQueue and cohorts

Kueue introduces the quota-based scheduling layer on top of Kubernetes (Kueue — Cluster Queue, Kueue — Cohort, Kueue — Fair Sharing):

Object	Scope	What it does
`ResourceFlavor`	cluster	maps resources to a group of nodes (e.g. H100 nodes)
`ClusterQueue`	cluster	defines `nominalQuota`, `borrowingLimit`, `lendingLimit` per resource/flavor
`LocalQueue`	namespace	entry point for the team’s workloads; points to a `ClusterQueue`
`Cohort`	cluster	groups ClusterQueues that can lend quota to each other

Quota concepts

nominalQuota: GPUs guaranteed to the ClusterQueue at all times.
borrowingLimit: maximum additional GPUs it can borrow from the cohort when others are not using them.
lendingLimit: GPUs from its nominalQuota that it allows to be lent to others (if not specified, it can lend all unused ones).
Fair Sharing: a mechanism that orders pending workloads by the historical resource usage of their LocalQueue, giving preference to whoever has consumed less. Compatible with hierarchical cohorts since Kueue v0.11.

Example YAML: 3 teams in the llm-platform cohort

# ResourceFlavor mapping to the H100 SXM nodes
apiVersion: kueue.x-k8s.io/v1beta2
kind: ResourceFlavor
metadata:
 name: h100-sxm
spec:
 nodeLabels:
 accelerator: h100-sxm
---
# ClusterQueue for the datos team — 2 guaranteed GPUs, can borrow up to 2
apiVersion: kueue.x-k8s.io/v1beta2
kind: ClusterQueue
metadata:
 name: cq-datos
spec:
 cohort: llm-platform
 namespaceSelector:
 matchLabels:
 kubernetes.io/metadata.name: ns-datos
 queueingStrategy: BestEffortFIFO
 resourceGroups:
 - coveredResources: ["nvidia.com/gpu"]
 flavors:
 - name: h100-sxm
 resources:
 - name: "nvidia.com/gpu"
 nominalQuota: 2
 borrowingLimit: 2 # can use up to 4 GPUs in total
 preemption:
 reclaimWithinCohort: LowerPriority
 withinClusterQueue: LowerPriority
---
# ClusterQueue for the ia team — 1 guaranteed GPU, lends what it does not use
apiVersion: kueue.x-k8s.io/v1beta2
kind: ClusterQueue
metadata:
 name: cq-ia
spec:
 cohort: llm-platform
 namespaceSelector:
 matchLabels:
 kubernetes.io/metadata.name: ns-ia
 resourceGroups:
 - coveredResources: ["nvidia.com/gpu"]
 flavors:
 - name: h100-sxm
 resources:
 - name: "nvidia.com/gpu"
 nominalQuota: 1
 lendingLimit: 1 # lends its GPU when it is not using it
---
# ClusterQueue for the plataforma team — 1 guaranteed GPU, low priority
apiVersion: kueue.x-k8s.io/v1beta2
kind: ClusterQueue
metadata:
 name: cq-plataforma
spec:
 cohort: llm-platform
 namespaceSelector:
 matchLabels:
 kubernetes.io/metadata.name: ns-plataforma
 resourceGroups:
 - coveredResources: ["nvidia.com/gpu"]
 flavors:
 - name: h100-sxm
 resources:
 - name: "nvidia.com/gpu"
 nominalQuota: 1
 borrowingLimit: 3 # can use all 4 if the rest are free
---
# LocalQueue in each namespace — entry point for the workloads
apiVersion: kueue.x-k8s.io/v1beta2
kind: LocalQueue
metadata:
 name: lq-datos
 namespace: ns-datos
spec:
 clusterQueue: cq-datos

How Kueue materialises the “GPU budget”

FinOps concept	Kueue mechanism
Guaranteed budget	`nominalQuota`: the team always has these GPUs available
Maximum spend limit	`nominalQuota + borrowingLimit`: absolute ceiling of admissible GPUs
Lending idle capacity	cohort + `lendingLimit`: others take what this one does not use
Fair sharing between teams	`Fair Sharing` + `WorkloadPriorityClass`: those who have consumed most wait longest
Reclaiming your own quota	`preemption.reclaimWithinCohort: LowerPriority`: the owner of the lent GPU reclaims it by evicting lower-priority workloads

The nominalQuota is the expression of the budget in GPUs: if the team has 2 nominal GPUs and the price is 1.40 €/GPU-hour, its maximum guaranteed spend is

\[ \text{maximum nominal spend} = 2 \times 1.40 \times 720 = 2{,}016 \text{ €/month} \]

The borrowingLimit sets the potential overspend if it borrows from the cohort (and that borrowing can be charged with the same formula, multiplying the borrowed usage hours by 1.40 €/GPU-hour).

Cost model: an example with 4×H100 SXM and 3 teams

Node price

Generic hardware: a server with 4×H100 SXM 80 GB. 2026 market prices: a single H100 SXM costs between 27,000 and 40,000 € depending on the source (GMI Cloud — H100 GPU Pricing 2026); a 4×H100 server sits in the 120,000–180,000 € range including chassis, power supplies and NVLink. We use 140,000 € as a generic assumption for the complete server (4 GPUs plus infrastructure).

Component	Calculation	€/hour node
Depreciated capex (140,000 €, 4 years, 90 % availability)	140,000 ÷ (4 × 8,760 × 0.9)	~4.43
Energy (4 × 700 W TDP × PUE 1.4 × 0.12 €/kWh)	4 × 0.7 × 1.4 × 0.12	~0.47
Operations / networking / maintenance	estimate	~0.70
Total 4×H100 node		~5.60 €/h
Per GPU	5.60 ÷ 4	~1.40 €/GPU-hour

Cost-per-token formula for the 70B model (sustained throughput ~2,000 tok/s at TP=4):

$$\text{€/1M tokens} = \frac{5.60 \text{ €/h}}{2{,}000 \text{ tok/s} \times 3{,}600 \text{ s/h}} \times 10^6 \approx 0.78 \text{ €/1M tokens}$$

Monthly scenario: 3 teams, heterogeneous utilisation

Reference month (720 hours). The 4 GPUs are nominally assigned: 2 to Datos, 1 to IA, 1 to Plataforma. Real occupancy varies:

Team	Nominal GPUs	Average occupancy	GPU-hours used	Idle GPU-hours	GPU cost (€)
Datos	2	75 %	1,080	360	~1,512
IA	1	60 %	432	288	~605
Plataforma	1	35 %	252	468	~353
Cluster idle	—	—	—	1,116	~1,562
Node total	4	—	1,764	1,116	~4,032

Total node cost per month: 5.60 €/h × 720 h = 4,032 €. Cost per GPU-hour used: 5.60 ÷ 4 = 1.40 €.

With shareIdle=false (showback), the 1,562 € of idle appears as a separate row in the report and nobody pays for it directly. With shareIdle=true (proportional chargeback), it is split among the three in proportion to their non-idle cost: Datos absorbs ~857 €, IA ~333 €, Plataforma ~192 € of additional idle.

Token consumption per team (LiteLLM spend logs)

Team	Model	Tokens/month	Cost/token	Token cost (€)
Datos	llama-70b (TP=4)	380 M	0.78 €/1M	~296
IA	llama-8b (1 GPU)	210 M	0.35 €/1M	~74
Plataforma	llama-8b + batch	80 M	0.35 €/1M	~28

Monthly chargeback report (showback + tokens)

Team	Iron GPU cost (€)	Assigned idle cost (€)	LiteLLM token cost (€)	Total chargeable (€)
Datos	1,512	857	296	2,665
IA	605	333	74	1,012
Plataforma	353	192	28	573
Idle (without shareIdle)	1,562	—	—	visible, not charged

Attribution dimensions × policy table

Attribution dimension	Tool	Showback	Chargeback
namespace	OpenCost `aggregate=namespace`	`/allocation?aggregate=namespace&includeIdle=true`	`shareIdle=true` + transfer to P&L
pod label	OpenCost `aggregate=label:equipo`	report per label	same with shareIdle
virtual key / team	LiteLLM `/global/spend/teams`	token dashboard	hard `max_budget` per key
ClusterQueue (reserved GPU)	Kueue `nominalQuota`	observe usage vs quota	committed budget in GPUs
request tag	LiteLLM `tag_budgets`	per cost centre	budget per tag
pod annotation	OpenCost `aggregate=annotation:KEY`	report per annotation	same with shareIdle

End-to-end data flow

When the Datos team has exhausted its 2 nominal GPUs and borrows those of Plataforma (which is not using them), Kueue records that loan. As soon as Plataforma launches a new workload:

Kueue detects that cq-plataforma is below its nominalQuota.
With preemption.reclaimWithinCohort: LowerPriority, Kueue evicts the Datos workload that was on the borrowed GPU, if its priority is lower.
The evicted workload returns to the cq-datos queue and is readmitted once quota frees up.

Fair Sharing orders the pending queue by the LocalQueue’s accumulated historical usage: if Datos has consumed more GPU-hours than IA in the recent period, IA’s next workloads have admission priority over new ones from Datos. This implements an equitable split without blocking anyone permanently.

Configuring OpenCost for the node price

For the numbers in the previous section to be reproducible, the node price has to be declared explicitly (the default value underestimates on-prem GPU, issue #3781):

# OpenCost Helm values.yaml — on-prem 4×H100 node, 5.60 €/h
opencost:
 customPricing:
 enabled: true
 provider: custom
 costModel:
 description: "On-prem 4xH100 SXM node"
 CPU: "0.025" # €/CPU-hour (minor component)
 RAM: "0.003" # €/GB-hour
 GPU: "1.40" # €/GPU-hour ← the one that drives the split
 storage: "0.0002" # €/GB-hour

Verify after configuring:

# Check the price resolved per node
curl http://localhost:9003/allNodePricing

# Attribution query per team label, last month, idle separate
curl -G http://localhost:9003/allocation \
 -d window=lastmonth \
 -d aggregate=label:equipo \
 -d includeIdle=true \
 -d shareIdle=false \
 -d accumulate=true

Reference PromQL for chargeback dashboards

# GPU cost assigned per team (label:equipo) — €/hour
sum by (label_equipo) (
 container_gpu_allocation
 * on(node) group_left node_gpu_hourly_cost
)

# Idle GPU-hours per node (idle > 15 min)
sum by (node) (
 (1 - avg_over_time(DCGM_FI_DEV_GPU_UTIL[15m]) / 100)
 * on(node) group_left node_gpu_hourly_cost
)

# Accumulated monthly cost per team (sum over window)
sum_over_time(
 sum by (label_equipo) (
 container_gpu_allocation * on(node) group_left node_gpu_hourly_cost
 )[30d:1h]
)

Sources

FinOps Foundation — Invoicing & Chargeback Capability — https://www.finops.org/framework/capabilities/invoicing-chargeback/
FinOps Foundation — Data Analysis and Showback — https://www.finops.org/framework/previous-capabilities/analysis-showback/
FinOps Foundation — Allocation Capability — https://www.finops.org/framework/capabilities/allocation/
OpenCost — API (Allocation API, parámetros window/aggregate/shareIdle/idleByNode) — https://opencost.io/docs/integrations/api/
OpenCost — API Examples — https://opencost.io/docs/integrations/api-examples/
OpenCost — GitHub (issue #3781, infra-precio GPU on-prem por defecto) — https://github.com/opencost/opencost/issues/3781
LiteLLM — Virtual Keys (max_budget, budget_duration, team_id) — https://docs.litellm.ai/docs/proxy/virtual_keys
LiteLLM — Setting Team Budgets — https://docs.litellm.ai/docs/proxy/team_budgets
LiteLLM — Spend Tracking — https://docs.litellm.ai/docs/proxy/cost_tracking
LiteLLM — Budgets & Rate Limits — https://docs.litellm.ai/docs/proxy/users
LiteLLM — Setting Tag Budgets — https://docs.litellm.ai/docs/proxy/tag_budgets
LiteLLM — Budget Reset Times — https://docs.litellm.ai/docs/proxy/budget_reset_and_tz
Kueue — Cluster Queue (nominalQuota, borrowingLimit, lendingLimit, preemption) — https://kueue.sigs.k8s.io/docs/concepts/cluster_queue/
Kueue — Cohort — https://kueue.sigs.k8s.io/docs/concepts/cohort/
Kueue — Fair Sharing — https://kueue.sigs.k8s.io/docs/concepts/fair_sharing/
Kueue — Administer Cluster Quotas — https://kueue.sigs.k8s.io/docs/tasks/manage/administer_cluster_quotas/
GMI Cloud — NVIDIA H100 GPU Pricing 2026 — https://www.gmicloud.ai/en/blog/nvidia-h100-gpu-pricing-2026-rent-vs-buy-cost-analysis
IntuitionLabs — NVIDIA AI GPU Prices H100 Cost Guide — https://intuitionlabs.ai/articles/nvidia-ai-gpu-pricing-guide

Showback on lo0 — Tech Blog

GPU chargeback and showback in multi-tenancy: how to split cluster cost between teams

TL;DR

Showback vs chargeback: the FinOps Foundation definition

When to use each

GPU cost attribution with OpenCost

Allocation API: attribution parameters

Labelling inference pods

Shared costs and idle: the three modes

LiteLLM as the attribution point for token consumption

Virtual keys and teams

Configuring teams with a monthly budget

Endpoints for querying spend per team

Joining €/GPU-hour (iron) with tokens per team

Kueue as the GPU budget/quota mechanism

ClusterQueue, LocalQueue and cohorts

Quota concepts

Example YAML: 3 teams in the llm-platform cohort

How Kueue materialises the “GPU budget”

Cost model: an example with 4×H100 SXM and 3 teams

Node price

Monthly scenario: 3 teams, heterogeneous utilisation

Token consumption per team (LiteLLM spend logs)

Monthly chargeback report (showback + tokens)

Attribution dimensions × policy table

End-to-end data flow

Configuring OpenCost for the node price

Reference PromQL for chargeback dashboards

Sources

See also

Showback on lo0 — Tech Blog

GPU chargeback and showback in multi-tenancy: how to split cluster cost between teams

TL;DR

Showback vs chargeback: the FinOps Foundation definition

When to use each

GPU cost attribution with OpenCost

Allocation API: attribution parameters

Labelling inference pods

Shared costs and idle: the three modes

LiteLLM as the attribution point for token consumption

Virtual keys and teams

Configuring teams with a monthly budget

Endpoints for querying spend per team

Joining €/GPU-hour (iron) with tokens per team

Kueue as the GPU budget/quota mechanism

ClusterQueue, LocalQueue and cohorts

Quota concepts

Example YAML: 3 teams in the llm-platform cohort

How Kueue materialises the “GPU budget”

Cost model: an example with 4×H100 SXM and 3 teams

Node price

Monthly scenario: 3 teams, heterogeneous utilisation

Token consumption per team (LiteLLM spend logs)

Monthly chargeback report (showback + tokens)

Attribution dimensions × policy table

End-to-end data flow

Fair sharing and preemption: how Kueue reclaims quota

Configuring OpenCost for the node price

Reference PromQL for chargeback dashboards

Sources

See also