GPU chargeback and showback in multi-tenancy: how to split cluster cost between teams

Notation: amounts in euros (N €), decimals with a point. The dollar sign is not used (on this site it is a formula delimiter).

TL;DR

A cluster of 4×H100 SXM at ~1.40 €/GPU-hour (depreciated capex plus energy) shared between three teams with different occupancy produces a monthly chargeback report of 3 rows and an additional idle row that nobody claimed. Without tooling, that idle cost vanishes, diluted into the total. With OpenCost + LiteLLM + Kueue, attribution operates on three orthogonal planes: the iron (OpenCost, €/GPU-hour per namespace/label), token consumption (LiteLLM, €/token per key/team/model) and the scheduler quota (Kueue, GPUs reserved per ClusterQueue). Crossing the three produces the number that goes to finance: team B consumed X million tokens at Y €/1M tok, the GPU cost it Z €, and it still has 2 GPUs borrowed from the cohort that will cost it W € if it holds on to them next month.

Showback vs chargeback: the FinOps Foundation definition

The distinction is not one of technology but of accounting formality (FinOps Foundation — Invoicing & Chargeback, FinOps Foundation — Data Analysis and Showback):

Two points from the framework worth pinning down:

1. Showback is a requirement of any FinOps practice; chargeback is optional and depends on the organisation’s accounting policy supporting transfers between cost centres.
2. Neither is “more mature” than the other. The narrative that chargeback is the “grown-up” version is false according to the framework itself. The natural sequence is: showback → trust in the data → chargeback if policy allows it.

When to use each

GPU cost attribution with OpenCost

Allocation API: attribution parameters

The OpenCost /allocation API is the piece that turns Prometheus metrics into a cost report per dimension (OpenCost — API):

Query for a monthly report per team, with idle visible as a separate row:

curl -G http://localhost:9003/allocation \
  -d window=lastmonth \
  -d aggregate=label:equipo \
  -d includeIdle=true \
  -d shareIdle=false \
  -d resolution=10m

With shareIdle=false (the default), OpenCost returns idle as a separate __idle__ entry, which makes visible how much was paid for unused capacity. With shareIdle=true, that idle is distributed among the tenants in proportion to their non-idle cost: each absorbs its share of the waste, which is the right behaviour for a chargeback that penalises whoever is responsible for the idle.

Labelling inference pods

For aggregate=label:equipo to work, the inference pods need the right label. Two methods:

Label in the pod spec (the most direct):

# vLLM Deployment for the datos team
metadata:
  labels:
    equipo: datos
    producto: rag-prod
    modelo: llama-70b
    entorno: prod

Annotation (when the label is already taken by another convention):

# aggregate as an annotation
aggregate=annotation:finops.io/equipo

OpenCost supports aggregate=label:KEY and aggregate=annotation:KEY with the same syntax; the choice between the two depends on the cluster’s labelling convention.

Shared costs and idle: the three modes

For infrastructure with nodes dedicated per team, idleByNode=true is fairer: the idle of a node dedicated to team A is not transferred to team B.

LiteLLM as the attribution point for token consumption

Virtual keys and teams

LiteLLM materialises token chargeback with four mechanisms (LiteLLM — Virtual Keys, LiteLLM — Setting Team Budgets, LiteLLM — Spend Tracking):

Configuring teams with a monthly budget

# litellm-config.yaml — per-team budgets with the declared on-prem cost
model_list:
  - model_name: llama-3-70b-onprem
    litellm_params:
      model: openai/llama-3-70b
      api_base: http://vllm-svc:8000/v1
      input_cost_per_token:  0.00000140   # 1.40 €/1M tok (€/GPU-hour ÷ throughput)
      output_cost_per_token: 0.00000140
  - model_name: llama-3-8b-onprem
    litellm_params:
      model: openai/llama-3-8b
      api_base: http://vllm-8b-svc:8000/v1
      input_cost_per_token:  0.00000035   # 0.35 €/1M tok (1 GPU, higher throughput)
      output_cost_per_token: 0.00000035

litellm_settings:
  success_callback: ["langfuse"]   # or any external logger

# Teams (via the /team/new API or config)
# POST /team/new
# {
#   "team_alias": "equipo-datos",
#   "max_budget": 800,           # 800 € per month
#   "budget_duration": "30d",
#   "tpm_limit": 500000,         # max tokens/min
#   "rpm_limit": 1000            # max requests/min
# }

The input_cost_per_token key comes from the identity in article A4: the €/GPU-hour from OpenCost divided by the benchmark throughput. With that value, the response_cost of each request reflects the real cost of the on-prem iron for that model.

Endpoints for querying spend per team

# Team spend accumulated over the period
GET /team/info?team_id=equipo-datos

# Team spend per model (aggregated spend_logs table)
GET /global/spend/logs?team_id=equipo-datos&start_date=2026-06-01&end_date=2026-06-30

# Total spend across all teams
GET /global/spend/teams

The response includes spend (accumulated cost in the configured currency), total_tokens, prompt_tokens, completion_tokens and a per-model breakdown, the direct fields for the monthly chargeback report.

Joining €/GPU-hour (iron) with tokens per team

The two attribution planes, OpenCost (iron) and LiteLLM (tokens), are crossed with one key: the model_name and the namespace. The join happens outside the tools (in a BI pipeline or a SQL query over the LiteLLM database and the data exported from OpenCost):

coste_por_token_real = OpenCost.GPU_cost_namespace / LiteLLM.total_tokens_team

When the on-prem cost is properly declared in input_cost_per_token, LiteLLM already does this calculation internally and the response_cost is correct. The explicit join serves to validate: if the sum of LiteLLM’s response_cost per team does not match the GPU cost OpenCost assigns to the namespace, there is drift in input_cost_per_token that has to be corrected (changed throughput, a new optimisation, more replicas).

Kueue as the GPU budget/quota mechanism

ClusterQueue, LocalQueue and cohorts

Kueue introduces the quota-based scheduling layer on top of Kubernetes (Kueue — Cluster Queue, Kueue — Cohort, Kueue — Fair Sharing):

Quota concepts

• nominalQuota: GPUs guaranteed to the ClusterQueue at all times.
• borrowingLimit: maximum additional GPUs it can borrow from the cohort when others are not using them.
• lendingLimit: GPUs from its nominalQuota that it allows to be lent to others (if not specified, it can lend all unused ones).
• Fair Sharing: a mechanism that orders pending workloads by the historical resource usage of their LocalQueue, giving preference to whoever has consumed less. Compatible with hierarchical cohorts since Kueue v0.11.

Example YAML: 3 teams in the llm-platform cohort

# ResourceFlavor mapping to the H100 SXM nodes
apiVersion: kueue.x-k8s.io/v1beta2
kind: ResourceFlavor
metadata:
  name: h100-sxm
spec:
  nodeLabels:
    accelerator: h100-sxm
---
# ClusterQueue for the datos team — 2 guaranteed GPUs, can borrow up to 2
apiVersion: kueue.x-k8s.io/v1beta2
kind: ClusterQueue
metadata:
  name: cq-datos
spec:
  cohort: llm-platform
  namespaceSelector:
    matchLabels:
      kubernetes.io/metadata.name: ns-datos
  queueingStrategy: BestEffortFIFO
  resourceGroups:
    - coveredResources: ["nvidia.com/gpu"]
      flavors:
        - name: h100-sxm
          resources:
            - name: "nvidia.com/gpu"
              nominalQuota: 2
              borrowingLimit: 2    # can use up to 4 GPUs in total
  preemption:
    reclaimWithinCohort: LowerPriority
    withinClusterQueue: LowerPriority
---
# ClusterQueue for the ia team — 1 guaranteed GPU, lends what it does not use
apiVersion: kueue.x-k8s.io/v1beta2
kind: ClusterQueue
metadata:
  name: cq-ia
spec:
  cohort: llm-platform
  namespaceSelector:
    matchLabels:
      kubernetes.io/metadata.name: ns-ia
  resourceGroups:
    - coveredResources: ["nvidia.com/gpu"]
      flavors:
        - name: h100-sxm
          resources:
            - name: "nvidia.com/gpu"
              nominalQuota: 1
              lendingLimit: 1     # lends its GPU when it is not using it
---
# ClusterQueue for the plataforma team — 1 guaranteed GPU, low priority
apiVersion: kueue.x-k8s.io/v1beta2
kind: ClusterQueue
metadata:
  name: cq-plataforma
spec:
  cohort: llm-platform
  namespaceSelector:
    matchLabels:
      kubernetes.io/metadata.name: ns-plataforma
  resourceGroups:
    - coveredResources: ["nvidia.com/gpu"]
      flavors:
        - name: h100-sxm
          resources:
            - name: "nvidia.com/gpu"
              nominalQuota: 1
              borrowingLimit: 3   # can use all 4 if the rest are free
---
# LocalQueue in each namespace — entry point for the workloads
apiVersion: kueue.x-k8s.io/v1beta2
kind: LocalQueue
metadata:
  name: lq-datos
  namespace: ns-datos
spec:
  clusterQueue: cq-datos

How Kueue materialises the “GPU budget”

The nominalQuota is the expression of the budget in GPUs: if the team has 2 nominal GPUs and the price is 1.40 €/GPU-hour, its maximum guaranteed spend is

The borrowingLimit sets the potential overspend if it borrows from the cohort (and that borrowing can be charged with the same formula, multiplying the borrowed usage hours by 1.40 €/GPU-hour).

Cost model: an example with 4×H100 SXM and 3 teams

Node price

Generic hardware: a server with 4×H100 SXM 80 GB. 2026 market prices: a single H100 SXM costs between 27,000 and 40,000 € depending on the source (GMI Cloud — H100 GPU Pricing 2026); a 4×H100 server sits in the 120,000–180,000 € range including chassis, power supplies and NVLink. We use 140,000 € as a generic assumption for the complete server (4 GPUs plus infrastructure).

Cost-per-token formula for the 70B model (sustained throughput ~2,000 tok/s at TP=4):

Monthly scenario: 3 teams, heterogeneous utilisation

Reference month (720 hours). The 4 GPUs are nominally assigned: 2 to Datos, 1 to IA, 1 to Plataforma. Real occupancy varies:

Total node cost per month: 5.60 €/h × 720 h = 4,032 €. Cost per GPU-hour used: 5.60 ÷ 4 = 1.40 €.

With shareIdle=false (showback), the 1,562 € of idle appears as a separate row in the report and nobody pays for it directly. With shareIdle=true (proportional chargeback), it is split among the three in proportion to their non-idle cost: Datos absorbs ~857 €, IA ~333 €, Plataforma ~192 € of additional idle.

Token consumption per team (LiteLLM spend logs)

Monthly chargeback report (showback + tokens)

Attribution dimensions × policy table

End-to-end data flow

Fair sharing and preemption: how Kueue reclaims quota

When the Datos team has exhausted its 2 nominal GPUs and borrows those of Plataforma (which is not using them), Kueue records that loan. As soon as Plataforma launches a new workload:

1. Kueue detects that cq-plataforma is below its nominalQuota.
2. With preemption.reclaimWithinCohort: LowerPriority, Kueue evicts the Datos workload that was on the borrowed GPU, if its priority is lower.
3. The evicted workload returns to the cq-datos queue and is readmitted once quota frees up.

Fair Sharing orders the pending queue by the LocalQueue’s accumulated historical usage: if Datos has consumed more GPU-hours than IA in the recent period, IA’s next workloads have admission priority over new ones from Datos. This implements an equitable split without blocking anyone permanently.

Configuring OpenCost for the node price

For the numbers in the previous section to be reproducible, the node price has to be declared explicitly (the default value underestimates on-prem GPU, issue #3781):

# OpenCost Helm values.yaml — on-prem 4×H100 node, 5.60 €/h
opencost:
  customPricing:
    enabled: true
    provider: custom
    costModel:
      description: "On-prem 4xH100 SXM node"
      CPU: "0.025"      # €/CPU-hour (minor component)
      RAM: "0.003"      # €/GB-hour
      GPU: "1.40"       # €/GPU-hour  ← the one that drives the split
      storage: "0.0002" # €/GB-hour

Verify after configuring:

# Check the price resolved per node
curl http://localhost:9003/allNodePricing

# Attribution query per team label, last month, idle separate
curl -G http://localhost:9003/allocation \
  -d window=lastmonth \
  -d aggregate=label:equipo \
  -d includeIdle=true \
  -d shareIdle=false \
  -d accumulate=true

Reference PromQL for chargeback dashboards

# GPU cost assigned per team (label:equipo) — €/hour
sum by (label_equipo) (
  container_gpu_allocation
  * on(node) group_left node_gpu_hourly_cost
)

# Idle GPU-hours per node (idle > 15 min)
sum by (node) (
  (1 - avg_over_time(DCGM_FI_DEV_GPU_UTIL[15m]) / 100)
  * on(node) group_left node_gpu_hourly_cost
)

# Accumulated monthly cost per team (sum over window)
sum_over_time(
  sum by (label_equipo) (
    container_gpu_allocation * on(node) group_left node_gpu_hourly_cost
  )[30d:1h]
)

Sources

• FinOps Foundation — Invoicing & Chargeback Capability — https://www.finops.org/framework/capabilities/invoicing-chargeback/
• FinOps Foundation — Data Analysis and Showback — https://www.finops.org/framework/previous-capabilities/analysis-showback/
• FinOps Foundation — Allocation Capability — https://www.finops.org/framework/capabilities/allocation/
• OpenCost — API (Allocation API, parámetros window/aggregate/shareIdle/idleByNode) — https://opencost.io/docs/integrations/api/
• OpenCost — API Examples — https://opencost.io/docs/integrations/api-examples/
• OpenCost — GitHub (issue #3781, infra-precio GPU on-prem por defecto) — https://github.com/opencost/opencost/issues/3781
• LiteLLM — Virtual Keys (max_budget, budget_duration, team_id) — https://docs.litellm.ai/docs/proxy/virtual_keys
• LiteLLM — Setting Team Budgets — https://docs.litellm.ai/docs/proxy/team_budgets
• LiteLLM — Spend Tracking — https://docs.litellm.ai/docs/proxy/cost_tracking
• LiteLLM — Budgets & Rate Limits — https://docs.litellm.ai/docs/proxy/users
• LiteLLM — Setting Tag Budgets — https://docs.litellm.ai/docs/proxy/tag_budgets
• LiteLLM — Budget Reset Times — https://docs.litellm.ai/docs/proxy/budget_reset_and_tz
• Kueue — Cluster Queue (nominalQuota, borrowingLimit, lendingLimit, preemption) — https://kueue.sigs.k8s.io/docs/concepts/cluster_queue/
• Kueue — Cohort — https://kueue.sigs.k8s.io/docs/concepts/cohort/
• Kueue — Fair Sharing — https://kueue.sigs.k8s.io/docs/concepts/fair_sharing/
• Kueue — Administer Cluster Quotas — https://kueue.sigs.k8s.io/docs/tasks/manage/administer_cluster_quotas/
• GMI Cloud — NVIDIA H100 GPU Pricing 2026 — https://www.gmicloud.ai/en/blog/nvidia-h100-gpu-pricing-2026-rent-vs-buy-cost-analysis
• IntuitionLabs — NVIDIA AI GPU Prices H100 Cost Guide — https://intuitionlabs.ai/articles/nvidia-ai-gpu-pricing-guide

See also

• OpenCost in depth: how GPU cost is allocated in Kubernetes
• Kubecost vs OpenCost vs alternatives
• From GPU-hour to cost per token
• FinOps and GPU cluster multi-tenancy: who pays for what
• Sharing a GPU: time-slicing, MPS and MIG