The blog map: from the utility feed to the token served

Contents

This blog started in March 2026 with a handful of articles about Kubernetes and networking, and by now it has piled up more than 150 pieces that, read one at a time, look like unconnected topics: PagedAttention, diesel generators, OpenCost, SPIFFE, immersion cooling, RAGAS. They are not. All of them describe the same object from different altitudes: a sovereign inference factory, that is, the complete machine that turns megawatts from the grid into tokens served to a user, with ownership and control of the data kept in house.

This article is the index of that machine. It orders what has been published into ten layers, bottom up, and links what is written in each one. It is good for three things: finding a specific topic, seeing which layers are covered in depth and which are half done, and choosing a reading route according to the problem in front of you.

The map

The map reads bottom up, and the criterion for ordering it is physical dependency: there is no inference engine without a powered and cooled GPU, no RAG without an engine serving embeddings, no demonstrable regulatory compliance without observability recording what happened. Layer 10 is the exception, and that is why it appears cutting across everything: cost, performance and energy are not a floor of the building, they are the measure of the whole building.

Layer 0 · Physical infrastructure

The most recent vertical, and the one that closes the circle: the path of the watt from the utility feed to the chip, and the path of the heat from the chip to the atmosphere.

The power chain: redundancy, Tier, kW against kVA, PUE, AI density and the limit of the grid.
Generators: NFPA 110, transients, sizing, diesel against gas, fuel and paralleling.
Switching: ATS against STS, the ITIC window, open and closed transition, and the A and B paths.
The UPS and ride-through: IEC 62040-3 topologies, eco-mode, flywheel, lithium against lead and the pulsing loads of AI.
The thermal challenge: why AI density broke air, and the spectrum of solutions.
Air and its limits: CRAC and CRAH, the geometry of the flow, the cube law, containment, ASHRAE and rear doors.
Direct liquid to the chip: cold plate, CDU, TCS and FWS loops, coolant chemistry and leaks.
Immersion and the megawatt horizon: single-phase and two-phase, the PFAS wall and microfluidics in the silicon.

Layer 1 · The node

Inside the machine, before any process exists: the buses, the memory and the topology that decide how much performance actually reaches the model.

PCIe topology, GPUDirect and ACS: the corridors the tensors travel through.
NUMA, hugepages and CPU isolation: why the neighbour on your socket costs you latency.
NVLink, NVSwitch and NCCL: the cable a large model is split across.
Network NUMA with Cilium and DraNet: the kitchen door nobody watches.
CPU, Memory and Topology Manager in RKE2: how to ask for a GPU with affinity and no surprises.
SM, CUDA streams and CUDA graphs: where time is lost inside the accelerator.
Mixed NVIDIA and Intel environments: from the homogeneous cluster to the real estate.
Measuring the power of a GPU: sensor metrology, which belongs to this layer and to layer 10 at the same time.

Layer 2 · Storage

State of the art, performance, security and availability: the set of four on the storage that holds weights, corpus and checkpoints.
From disk to HBM: why starting a model takes as long as it takes.
Speeding up the cold start: from minutes to seconds with serialisation and cache.

Layer 3 · The cluster

vLLM on Kubernetes: the inference piece that does scale.
LLM inference operators: OME, vLLM Production Stack and company.
The GPU cluster as a multi-tenant platform: from shared machine to service.
Sharing a GPU: time-slicing, MPS and MIG: the three ways to split an accelerator.
Volcano and Kueue: gang scheduling, queues and quotas.
Autoscaling with HPA and KEDA: scaling on metrics that mean something.
Serving several models on one GPU: co-residency, swap and sleep.
On-premise capacity planning: from the SLO to the number of nodes.
GitOps for the stack with Flux: operating the platform as code.
Cilium BGP on RKE2 and eBPF from zero to Cilium: the network underneath everything above.

Layer 4 · The inference engine

The densest block of the blog: vLLM cut open, piece by piece.

KV cache and PagedAttention: the working memory and its block manager.
Continuous batching and the scheduler step: how each round of work is assembled.
Optimising prefill and optimising decode: the two phases and their controls.
The attention backend and FlashAttention: the core of the computation.
Prefix cache: hit rate engineering and batch sizing with grid search: the two settings that give away the most performance.
Disaggregated serving and one big replica or many small ones: the topology decisions.
Long context and KV offloading: the KV memory hierarchy when the window does not fit in HBM.
Structured output and function calling: holding the model’s output in place.
Serving reasoning models: the invisible draft you pay for all the same.
Multimodal on-premise with vLLM and serving embeddings and rerankers with TEI: the other models in the stack.
The inverted roofline: why a small model is not optimised the same way as a large one.

Layer 5 · The model

Everything done to the model before serving it, or while it is being served.

Quantization for inference, FP8 end-to-end, aggressive sub-4-bit quantisation and quantising on the fly.
Pruning and distillation: reducing without amputating, and teaching a small model.
MoE in inference and device-native architectures.
Multi-LoRA serving, QLoRA and multi-LoRA at the limit and the complete QLoRA runbook.
Speculative decoding and self-speculative with early exit.
Modern alignment: DPO, KTO, ORPO and SimPO.
Continuous fine-tuning and retrain: closing the loop.

Layer 6 · Data and RAG

Document ingestion from the PDF to the indexed chunk and corpus curation.
Embeddings in 2026 and reranker with hybrid retrieval.
PostgreSQL and Qdrant in ingestion, Debezium and CDC and RAG over Kafka.
Semantic cache: the receptionist with a memory.
RAG on CPU and aggressive RAG on small models.
Evaluating a RAG without fooling yourself: RAGAS and the golden dataset.
Ontologies and knowledge graphs and data versioning with DVC and lakeFS.

Layer 7 · Control plane and delivery

Choosing the OSS gateway and the L7 inference router: the switchboard in front of the engine.
KServe and the Open Inference Protocol: the standard control plane.
Model registry and distribution with OCI and ORAS: where the bytes live.
Canary, blue-green and shadow: deploying a model without breaking the service.
Prompt versioning: the contract that avoids the scare.
MCP from the inside and adding authentication with Keycloak.
Operating the control plane (September 2026 run): the LiteLLM and Langfuse pair, day 2 of the proxy and virtual keys, budgets and limits and humans and agents on the same gateway, the prefix routing LiteLLM does not do and the MCP gateway as a second front door, sizing for agents and the seams with identity and traces: the gateway no longer as a piece you choose, but as a piece you operate.
Identity: Keycloak in an AI platform, with its position in the architecture and the MCP standards it does not implement, and how to complete it for MCP by building the protected resource side.
Operating Langfuse (a run of eight, September 2026): what goes into a trace, the first article about day 2 of version 4.
Building an end-to-end sovereign assistant, with all of the above pieces working together, is the exercise that closes this layer and is pending publication.

Delivery to the teams that use it is its own family of pieces:

Backstage as a self-service portal: the catalogue, the templates and the golden path, with the real cost of operating it.
Kubeflow in depth: which pieces of the ML toolbox are worth it and which ones you already have.
Knative and scale-to-zero: when switching off the GPU saves money and when the cold start violates your SLO.

Rebuilding the cloud experience on your own iron is a run of three:

From public cloud to private: what you lose when you leave the cloud and which projects fill each gap, with the business case and the sovereignty drivers.
Crossplane as a control plane: the declarative provisioning API that on-premise does not give you, and the cost of operating your own control plane.
KubeVela and Score: application-centric self-service by CLI, the friendly face Crossplane deliberately does not offer.

Layer 8 · Observability and evaluation

Tracing with OpenTelemetry GenAI and instrumenting vLLM with OTel.
Langfuse from the inside: the sorting centre for traces.
GPU observability with DCGM and the anatomy of DCGM and vLLM metrics.
Evals: the layer after tracing and LLM-as-judge.
Incident response runbooks: from the alert to the action.
Hubble, AgentSight and drift detection with eBPF: observing without instrumenting.

Layer 9 · Security, trust and compliance

Guardrails and safety and LLM Guard: the lines of defence around the model.
Hardening and secrets of the sovereign stack.
Isolating AI agents from the workspace and the runbook with bubblewrap and Tetragon, with Tetragon as the base.
Signing, provenance and AIBOM and identity and isolation with SPIFFE and Confidential Containers.
ISO/IEC 42001, the EU AI Act article by article and the cross-mapping of ENS, 42001 and the AI Act.
Compliance in healthcare and in defence: the same infrastructure facing two sector frameworks.

Layer 10 · Cost, performance and energy

The data track, written in a different format from the rest of the blog: tables, figures and a verifiable source, no analogies. These are the articles that hold up an investment decision.

FinOps: cost model and state of the art, OpenCost in depth, Kubecost against OpenCost, from the GPU-hour to the cost per token, chargeback and showback, the cost of the idle GPU, cloud GPU and neoclouds, full on-premise TCO and multi-tenancy FinOps with LiteLLM.

Benchmarking: frameworks and metrics, the tool catalogue, GuideLLM and the SLO under load, GenAI-Perf in depth, how to read MLPerf Inference, measurement bias and reproducibility, quality benchmarks and the Pareto frontier between engines.

Energy: measuring GPU power, energy per token in Spain, energy benchmarking frameworks, measuring in production with Kepler and DCGM, MLPerf Power, Wh per token leaderboards, from the watt to carbon, efficiency levers and energy in the TCO and in regulation.

Synthesis: the three axes and the identity that binds them, sovereign on-premise against hyperscalers, with data, from the SLO to the number of GPUs and the reproducible harness.

To that list you can add two cost pieces that are not about GPUs: the cost of agents with durable execution and the gateway’s cost per request.

The master maps

Five earlier articles work as the general plan and are still the best way in for anyone arriving new:

The six-stage LLMOps pipeline: the master map of the life cycle.
The seven layers of the on-premise stack: the architecture version of this same map.
Anatomy of an LLM request in production: a real request crossing every layer.
Five maturity levels of the platform and seven greenfield deployment phases: where you are and what comes next.
The OSS catalogue entry by entry and the parallel catalogue against the hyperscalers: which tool for each stage.
MLOps for LLMs in 2026: the starting landscape.

Four routes

I am about to build the first platform. Landscape and the six-stage pipeline, the seven layers of the stack, the seven deployment phases, capacity planning, vLLM on Kubernetes, gateway, observability with OTel and Langfuse, and guardrails before opening the door.

I have the cluster and the performance is not there. Anatomy of a request, KV cache and PagedAttention, prefill and decode, prefix cache, batch sizing, GuideLLM to find the saturation point, measurement bias so you do not fool yourself with the numbers, and the Pareto frontier between engines.

I have to justify the investment to management. The three axes, on-premise TCO, cost per token, idle GPU, on-premise against hyperscalers, sizing and justifying, and the reproducible harness so the figures survive an audit.

My job is the room, not the software. The power chain, the generators, the thermal challenge, air and its limits, direct liquid to the chip, immersion, and from there to storage and node topology.

What is missing

The map also serves to show the gaps, and three are identified, plus a line that is half open:

A comparison of vector databases: Milvus, Qdrant, pgvector and OpenSearch with data, beyond the scattered mentions.
Lakehouse for RAG: Iceberg and Trino as the substrate of the corpus.
Out-of-band management: OpenBMC, Redfish and DCIM, which is exactly the bridge between layer 0 and layer 3.
Sovereign substrate: the CLI self-service part is already covered by the layer 7 run (Crossplane, KubeVela and Score), and what remains is the IaaS-type substrate underneath, the sovereign alternatives to the hyperscaler (Spinifex, Harvester, OpenStack) on which everything else runs.