# lo0 — Tech Blog > Technical blog on networking, cybersecurity, sovereign infrastructure, MLOps and LLMs in production. Every article published in Spanish and English. Field notes from the lo0 instructors on networking, Kubernetes, clustering, MikroTik, Ubiquiti, Huawei, artificial intelligence, LLMs in production, sovereign MLOps and IT certifications. Versión en castellano: https://blog.lo0.es/llms.txt ## Pages - [About this blog](https://blog.lo0.es/en/about/index.md) ## Ai - [The GPU cluster as a platform: turning a shared cluster into a multi-tenant service your teams can consume](https://blog.lo0.es/en/posts/gpu-cluster-as-multi-tenant-platform/index.md): How to turn a shared GPU cluster into an internal multi-tenant platform: the AI gateway layer (LiteLLM, Portkey, Kong AI Gateway), GPU isolation (MIG vs MPS vs time-slicing), … - [RAG on Kafka: a technical reference architecture for streaming datalakes, with fresh embeddings and vector stores that are always up to date](https://blog.lo0.es/en/posts/rag-on-kafka-reference-architecture-streaming-datalakes/index.md): The second post in the MLOps for LLMs series. End-to-end RAG architecture on Kafka: an immutable log as source of truth, Debezium CDC from PostgreSQL, Flink SQL with embedding … - [eBPF in local inference and statistical drift detection: closing the LLM observability loop in 2026](https://blog.lo0.es/en/posts/ebpf-local-inference-statistical-drift-detection/index.md): The close of the post-tracing and eBPF series. How eBPF observes local inference without instrumenting the code (uprobes on llama.cpp, vLLM, libcudart), the ProfInfer 2026 paper, … - [MCP from the inside and its deep observability: the LSP of AI agents and how to see everything with OpenTelemetry](https://blog.lo0.es/en/posts/mcp-deep-observability-opentelemetry/index.md): Model Context Protocol explained with analogies (the USB-C of AI apps, the LSP of editors, the operating system driver): what problem it solves, the complete architecture (Host, … - [AgentSight and the new LLM tracing: zero-instrumentation with eBPF against Langfuse, LangSmith, Phoenix and company](https://blog.lo0.es/en/posts/agentsight-llm-tracing-ebpf-zero-instrumentation/index.md): Fourth and last post of the eBPF series. How an LLM agent is observed in 2026: the classic instrumented approach (Langfuse, LangSmith, Arize Phoenix, OpenLLMetry, Pydantic Logfire) … - [LLM inference operators on Kubernetes: OME, vLLM Production Stack, NVIDIA Dynamo and llm-d](https://blog.lo0.es/en/posts/llm-inference-operators-kubernetes/index.md): The fourth post in the inference series. Why a flat Deployment is no longer enough, what CRDs the four dominant operators bring in 2026 (OME, vLLM Production Stack, NVIDIA Dynamo, … - [vLLM on Kubernetes: the LLM inference piece that actually scales](https://blog.lo0.es/en/posts/vllm-kubernetes-llm-inference-that-scales/index.md): How vLLM works inside (PagedAttention, continuous batching, scheduler), why Kubernetes is its natural habitat, what it takes to deploy it seriously (GPU operator, tensor parallel, … ## Ai infrastructure - [Storage in the AI era (1/4): the state of the art](https://blog.lo0.es/en/posts/ai-era-storage-state-of-the-art/index.md): First article in the series on storage for AI: the memory-storage hierarchy, the memory wall, NVMe Gen6, 245 TB QLC, CXL, parallel file systems and the new inference stack with … - [Storage in the AI era (2/4): performance](https://blog.lo0.es/en/posts/ai-era-storage-performance/index.md): Second article in the series: real performance metrics (GB/s, IOPS, latency), MLPerf Storage v2.0, GPUDirect Storage, the I/O pattern of training, the cost of LLM checkpointing and … - [Storage in the AI era (3/4): security](https://blog.lo0.es/en/posts/storage-ai-era-security/index.md): Third article in the series: encryption at rest and in transit, the transition to FIPS 140-3, post-quantum cryptography and 'harvest now, decrypt later', data poisoning, … - [Storage in the AI era (4/4): availability](https://blog.lo0.es/en/posts/storage-ai-era-availability/index.md): Fourth and final article in the series: redundancy with erasure coding versus RAID, eleven nines of durability, real Backblaze AFRs, synchronous replication and RPO/RTO, resilience … ## Fundamentals - [Langfuse v4, day 2 (5 of 10): the worker's thirty-nine queues, and the three that decide whether you lose data](https://blog.lo0.es/en/posts/langfuse-v4-worker-queues/index.md): Fifth article in the series about operating Langfuse v4. The worker consumes thirty-nine BullMQ queues with thirty-two switches, different concurrencies and limiters ranging from … - [Langfuse v4, day 2 (4 of 10): migrating from 3 to 4 with no window, and where the point of no return sits](https://blog.lo0.es/en/posts/langfuse-v4-migrating-with-no-window/index.md): Fourth article in the series about operating Langfuse v4. The migration is online and needs no downtime, but the default write mode is the destination rather than the origin, and … - [Langfuse v4, day 2 (3 of 10): two well-known agents instrumented, Open Deep Research and GPT Researcher](https://blog.lo0.es/en/posts/langfuse-v4-real-agents-odr-gptr/index.md): Third article in the series about operating Langfuse v4. The previous one measured a toy graph; this one takes two open source agents people actually deploy, Open Deep Research and … - [Langfuse v4, day 2 (2 of 10): putting LangGraph in front, and what one agent turn costs in observations](https://blog.lo0.es/en/posts/langfuse-v4-instrumenting-langgraph/index.md): Second article in the series about operating Langfuse v4. The series grows to ten, because before talking about migration and capacity there has to be a real agentic platform in … - [Completing Keycloak for MCP: the protected resource no SDK hands you ready-made, and the enterprise extension that skips the whole flow](https://blog.lo0.es/en/posts/completing-keycloak-for-mcp-protected-resource/index.md): A follow-up to the Keycloak post. The authorisation server does not implement resource indicators and considers protected resource metadata out of its scope, so the whole load … - [Keycloak in an AI platform: where it fits, what it solves and the two standards MCP requires and it does not implement](https://blog.lo0.es/en/posts/keycloak-identity-plane-ai-platform/index.md): An introduction to Keycloak and its place in an on-premise LLM inference platform. What it is as of September 2026, how it is built, the conceptual model you need to understand … - [Langfuse v4, day 2 (1 of 10): the data model changed underneath, and what really goes into a trace](https://blog.lo0.es/en/posts/langfuse-v4-what-goes-into-a-trace/index.md): First article in a series of ten about operating Langfuse v4 in production. Version 4 no longer has a trace table: a trace is now a group of correlated observations, with the trace … - [The gateway does not live alone: the JWT that does not validate the audience, the user who never reaches the trace, and the seams you have to write by hand](https://blog.lo0.es/en/posts/gateway-jwt-audience-trace-identity-seams/index.md): Eighth article in the operational track of the control layer. The gateway connects to an identity provider and to a trace backend, and those two seams carry more work than the … - [Sizing for agents: 275 input tokens for every output token, and why the bottleneck is still decode](https://blog.lo0.es/en/posts/sizing-agents-decode-bottleneck/index.md): Seventh article in the operational track of the control layer. The previous one on agents dealt with coexistence; this one deals with size. With the first production-scale … - [LiteLLM's MCP gateway: the second front door, the tool catalogue nobody bills, and why filtering it can make selection worse](https://blog.lo0.es/en/posts/litellm-mcp-gateway-second-front-door/index.md): Sixth post in the operational track on the control layer. The same proxy that hands out tokens is also an MCP server, and that second door has rules of its own. This post walks … - [Prefix routing: what LiteLLM does not do, who does, and how much it really improves](https://blog.lo0.es/en/posts/prefix-routing-what-litellm-does-not-do/index.md): Fifth article in the operational track of the control layer. In June I wrote that LiteLLM had prefix-aware routing in its enterprise version. That was false, and this article … - [Humans and agents on the same gateway: why `priority` does not prioritise, the 429 that brings down the whole pool, and the two ways to separate traffic](https://blog.lo0.es/en/posts/humans-agents-same-gateway-priority-traffic-separation/index.md): Fourth article in the operational track of the control layer. What happens the day an agent starts calling the same gateway as the humans: why LiteLLM's priority parameter does not … - [Virtual keys, budgets and limits in LiteLLM: the layer that decides who gets the GPU, and the four things the documentation gets wrong](https://blog.lo0.es/en/posts/litellm-virtual-keys-budgets-limits/index.md): LiteLLM Proxy access control as of September 2026, read from the code and not from the documentation: the organisation, team, user and end-customer hierarchy and the rule that … - [LiteLLM on day 2: one worker per pod, six thousand seconds of timeout and the other five things to change before opening the traffic](https://blog.lo0.es/en/posts/litellm-day-2-high-availability/index.md): The gateway is installed with a ConfigMap and operated for years. This post walks through day 2 of LiteLLM Proxy on Kubernetes: the process model and why one worker per pod, … - [LiteLLM and Langfuse: the operational pair, and the four places where the trace is lost between the gateway and the dashboard](https://blog.lo0.es/en/posts/litellm-langfuse-operational-pair/index.md): The gateway and the tracing backend are installed in one afternoon and operated for years. This post walks through the seam between LiteLLM and Langfuse as of September 2026: the … - [Knative and scale-to-zero for LLM inference: when switching off the GPU saves money and when it costs you the SLO](https://blog.lo0.es/en/posts/knative-scale-to-zero-llm-inference/index.md): Knative scales a service to zero in seconds, but an LLM takes from tens of seconds to minutes to load its weights from disk into HBM. This article separates the two things: how … - [KubeVela and Score: handing over the platform by CLI without teaching the developer Kubernetes](https://blog.lo0.es/en/posts/kubevela-score-platform-cli-without-teaching-kubernetes/index.md): KubeVela and Score cover the layer Crossplane leaves out: the application-centric developer experience, by CLI. KubeVela uses the OAM model and a friendly CLI where your platform … - [Kubeflow in depth: which pieces are worth it in an on-premise LLM platform and which ones you already have](https://blog.lo0.es/en/posts/kubeflow-pieces-worth-it-on-premise-llm-platform/index.md): Kubeflow graduated in the CNCF in August 2026 and stopped being a monolith to become a collection of independent subprojects. This post takes them apart one by one (Pipelines, … - [Crossplane: the control plane that turns your cluster into a private cloud API](https://blog.lo0.es/en/posts/crossplane-control-plane-private-cloud-api/index.md): Crossplane graduated from the CNCF in November 2025 and proposes something different from Terraform: instead of applying infrastructure on demand, it reconciles it continuously as … - [From public cloud to private: what you really lose and which projects fill the gap](https://blog.lo0.es/en/posts/public-cloud-to-private-what-you-lose-which-projects-fill-gap/index.md): Repatriating workloads from the public cloud to on-premise saves money and gives you back control of your data, but the cloud does not only sell compute: it sells operations … - [Backstage as a self-service portal: giving your teams the LLM platform without giving them the cluster](https://blog.lo0.es/en/posts/backstage-self-service-portal-llm-platform/index.md): Backstage is the most widely adopted developer portal, but it is not a product: it is a Node/React framework that you compile and maintain as your own code, at a cost of two to … - [The blog map: from the utility feed to the token served](https://blog.lo0.es/en/posts/blog-map-from-utility-feed-to-token-served/index.md): A conceptual map of the blog: every article published up to August 2026, ordered by the ten layers that separate the utility feed from the token a user receives. Physical … - [MLPerf Inference: how to read it, what LoadGen enforces and what comparability it offers](https://blog.lo0.es/en/posts/mlperf-inference-reading-loadgen-comparability/index.md): MLPerf Inference is the only inference benchmark with written rules, a mandatory load generator and an audit process. This data sheet documents the four scenarios, what LoadGen … - [Measuring the power of a GPU: NVML, DCGM and the sampling errors that invalidate your watts](https://blog.lo0.es/en/posts/measuring-gpu-power-nvml-dcgm-sampling-errors/index.md): A metrology sheet for measuring GPU energy: what nvmlDeviceGetPowerUsage returns exactly (on the H100 it is a one-second average, not instantaneous power), why the sensor only … - [Model chain of trust (1/4): KServe and the Open Inference Protocol, the control plane your serving is missing](https://blog.lo0.es/en/posts/kserve-open-inference-protocol-control-plane/index.md): Opens the series on the model chain of trust. What KServe brings (CNCF incubating since 29 September 2025, v0.19.0 on 14 June 2026) on top of a vLLM Deployment: InferenceService, … - [Model chain of trust (2/4): where the bytes live — registry, OCI artefacts and distribution to the GPU node](https://blog.lo0.es/en/posts/model-chain-of-trust-where-the-bytes-live-registry-oci-artefacts/index.md): Second article in the series: where the model's bytes come from and how they reach the GPU. The distinction between model registry, artefact repository and deployment storage; the … - [Model chain of trust (3/4): signature, provenance and AIBOM — why you trust those bytes](https://blog.lo0.es/en/posts/model-chain-of-trust-signature-provenance-aibom-why-you-trust-those-bytes/index.md): Third article in the series: signing model artefacts with Sigstore (cosign 3.1.1, Rekor v2), the OpenSSF Model Signing specification and its hash manifest, provenance with SLSA … - [Model chain of trust (4/4): who serves the model and which machine you trust](https://blog.lo0.es/en/posts/model-chain-of-trust-who-serves-the-model-which-machine-you-trust/index.md): Closing the series: cryptographic workload identity with SPIFFE/SPIRE (a CNCF graduated project, v1.15.1 from May 2026) and environment isolation with Kata Containers and … - [Volcano and Kueue: gang scheduling, queues and GPU quotas for distributed workloads on Kubernetes](https://blog.lo0.es/en/posts/volcano-kueue-gang-scheduling-gpu-quotas-kubernetes/index.md): The kube-scheduler cannot do gang scheduling or fair quotas between teams: a multi-GPU job starts partially and sits in deadlock waiting for the rest. Volcano (CNCF incubating) is … - [The reproducible harness: measuring cost, performance and energy in a single auditable experiment](https://blog.lo0.es/en/posts/reproducible-harness-cost-performance-energy-experiment/index.md): Closing piece of the data series: the complete design of the reproducible harness that captures €/1M tok (OpenCost/LiteLLM), Wh/token (DCGM/Kepler) and TTFT/ITL P99 … - [From the SLO to the number of GPUs: how to size and justify the investment in inference hardware](https://blog.lo0.es/en/posts/slo-to-number-of-gpus-sizing-justifying-inference-hardware/index.md): The complete sizing chain from demand to the number of 4×H100 SXM nodes: step-by-step formulas from the SLO (TTFT/ITL P99) to the required peak throughput, through TCO, €/1M tokens … - [Energy in TCO and in regulation: cost, PUE, CSRD and the new European reporting obligation for datacenters](https://blog.lo0.es/en/posts/energy-in-tco-and-regulation-cost-pue-csrd-eu-datacenter-reporting/index.md): Energy is between 6 % and 20 % of the opex of an on-premise GPU cluster, but 100 % of what the new European regulatory framework demands: EED 2023/1791, Delegated Regulation … - [Levers for spending fewer watts per token: a quantified catalogue for LLM inference](https://blog.lo0.es/en/posts/levers-fewer-watts-per-token-llm-inference/index.md): A quantified catalogue of energy efficiency levers in on-premise LLM inference: FP8/INT4/AWQ quantisation cuts J/token by 30–50%, power capping to 70% of TDP trims power by 30% … - [From watt to carbon, honestly: PUE, grid intensity and gCO₂ per token](https://blog.lo0.es/en/posts/watt-to-carbon-pue-grid-intensity-gco2-per-token/index.md): From energy per token to emissions per token: global PUE (average 1.54 according to Uptime Institute 2025), grid carbon intensity (Spain ~108 gCO₂/kWh in 2024, France ~20 gCO₂/kWh, … - [Measurement bias and reproducibility: why two benchmarks of the same model give figures that differ by up to 7×](https://blog.lo0.es/en/posts/measurement-bias-reproducibility-llm-benchmarks/index.md): An exhaustive catalogue of the sources of bias in LLM inference benchmarks, from single-process client saturation (a documented 7.2× discrepancy) to the tokenizer, the ISL/OSL … - [Full TCO of an on-premise GPU cluster: from capex to the all-in €/GPU-hour and the break-even against cloud](https://blog.lo0.es/en/posts/full-tco-on-premise-gpu-cluster-capex-gpu-hour-break-even/index.md): A full TCO model of an on-premise GPU cluster with 4×H100 SXM nodes: capex breakdown (GPUs, HGX baseboard, InfiniBand NDR network, NVMe, rack/PDU), opex (energy × PUE, staff, … - [Cloud GPU: on-demand, reserved and spot, and why neoclouds change the board (with 2026 prices)](https://blog.lo0.es/en/posts/cloud-gpu-on-demand-reserved-spot-neoclouds-2026-prices/index.md): A 2026 price table for H100/H200/B200 by provider and mode (on-demand, 1-3 year reserved, spot): neoclouds such as CoreWeave, Lambda, Nebius or Voltage Park against AWS, GCP and … - [LLM energy leaderboards: comparing models by Wh/token and choosing on efficiency](https://blog.lo0.es/en/posts/llm-energy-leaderboards-wh-per-token/index.md): Technical data sheet for the three OSS leaderboards of LLM energy efficiency: HF AI Energy Score (166 models, Wh/query, 1–5 stars on H100), ML.ENERGY Leaderboard v3 (J/token per … - [LLM quality benchmarks: the contamination trap and the OSS tools for not fooling yourself](https://blog.lo0.es/en/posts/llm-quality-benchmarks-contamination-trap-oss-tools/index.md): Why quality and speed are independent Pareto axes, how data contamination invalidates static leaderboards, and which OSS tools —lm-evaluation-harness, HELM, LiveBench, lighteval, … - [LLM serving engines compared on the Pareto frontier: vLLM, SGLang, TRT-LLM and Dynamo](https://blog.lo0.es/en/posts/llm-serving-engines-pareto-frontier-comparison/index.md): A comparison of the four OSS LLM serving engines in 2026: vLLM, SGLang, TensorRT-LLM and NVIDIA Dynamo. Capability matrix, throughput/latency figures on 4×H100 SXM FP8 (Llama-70B), … - [MLPerf Power: the standard energy-efficiency benchmark for on-premise ML systems](https://blog.lo0.es/en/posts/mlperf-power-energy-efficiency-benchmark/index.md): MLPerf Power, maintained by the MLCommons Power Working Group, is the only benchmark with certified measurement at the wall (SPEC PTDaemon + Yokogawa analyser) for comparing the … - [GPU idle: the cost that shows up on no invoice but is paid by the whole TCO](https://blog.lo0.es/en/posts/gpu-idle-cost-not-on-any-invoice/index.md): On an on-premise cluster of 4×H100 SXM, going from 20 % to 70 % useful occupancy cuts the cost per million tokens from ~18 € to ~5 €, without touching the hardware. This article … - [GPU chargeback and showback in multi-tenancy: how to split cluster cost between teams](https://blog.lo0.es/en/posts/gpu-chargeback-showback-multi-tenancy/index.md): How to split the cost of a 4×H100 SXM cluster between teams: FinOps definitions of showback vs chargeback, the OpenCost Allocation API with shareIdle/idleByNode, LiteLLM virtual … - [GenAI-Perf in depth: LLM inference profiling with real data](https://blog.lo0.es/en/posts/genai-perf-llm-inference-profiling-in-depth/index.md): A data deep dive into NVIDIA GenAI-Perf: metrics (TTFT, ITL/TPOT, P99 throughput), invocation flags against OpenAI-compatible endpoints (vLLM, NIM, TRT-LLM), concurrency sweep with … - [Sovereign on-premise vs hyperscalers: the case with data (cost, energy, performance and sovereignty)](https://blog.lo0.es/en/posts/sovereign-on-premise-vs-hyperscalers-data/index.md): Synthesis article of the series: the case for sovereign on-premise against the hyperscalers and the European cloud, crossing the four axes (cost/TCO and break-even, performance, … - [Measuring energy in production: Kepler, DCGM and the practical stack (accuracy and overhead)](https://blog.lo0.es/en/posts/measuring-energy-in-production-kepler-dcgm-stack/index.md): A practical deep dive into energy tooling: how the measurement stack is deployed on Kubernetes (DCGM exporter + Kepler eBPF + Prometheus + Grafana), which sources each piece takes … - [GuideLLM in depth: validating the SLO under load and sizing from the knee](https://blog.lo0.es/en/posts/guidellm-validating-slo-under-load-sizing-knee/index.md): A data deep dive into GuideLLM: the rate-types (synchronous, concurrent, poisson, throughput, sweep), the automatic sweep that finds the knee, how to define the SLO, the output … - [The second cost vector of AI agents: durable execution with Temporal](https://blog.lo0.es/en/posts/second-cost-vector-ai-agents-durable-execution-temporal/index.md): Agents have two cost vectors: the inference one (which local vLLM eliminates) and the one from faulty execution (failures charged twice, uncontrolled retries, inflated context on … - [From GPU-hour to cost per token: the metric that compares on-prem and cloud](https://blog.lo0.es/en/posts/gpu-hour-to-cost-per-token-on-prem-vs-cloud/index.md): A data deep dive: how the cost per token and per request of your own inference is calculated, why it is not a list price but €/GPU-hour divided by throughput, how to instrument it … - [Energy per token in Spain: methodology, the electricity market and the cases of speculation](https://blog.lo0.es/en/posts/energy-per-token-spain-methodology-electricity-market-speculation/index.md): A deep dive focused on Spain: how energy per token is measured, how the electricity price is formed in OMIE's marginal-pricing market, the documented cases of speculation and … - [Kubecost vs OpenCost vs the alternatives: what the commercial layer adds and when it is worth paying for](https://blog.lo0.es/en/posts/kubecost-vs-opencost-vs-alternatives/index.md): A data deep dive: what Kubecost (IBM) adds on top of free OpenCost (invoice reconciliation, rightsizing, anomalies, RBAC, multi-cluster), the commercial alternatives (CloudZero, … - [OpenCost in depth: how GPU cost is allocated in Kubernetes](https://blog.lo0.es/en/posts/opencost-gpu-cost-allocation-kubernetes/index.md): A data deep dive into OpenCost: the node-level cost model, where it gets its data from (Prometheus, DCGM), how on-prem prices are configured in euros, allocation by usage (not by … - [LLM benchmark tool catalogue: the practical deep dive](https://blog.lo0.es/en/posts/llm-benchmark-tool-catalogue-practical-deep-dive/index.md): A practical deep dive into LLM inference benchmarking tooling: what each tool measures, how it is invoked (real commands), what output format it produces and when to choose which. … - [Cost, performance and energy: the three axes of LLM inference and the identity that ties them together](https://blog.lo0.es/en/posts/cost-performance-energy-three-axes-llm-inference/index.md): Opening article of the data series on FinOps, LLM benchmarking and energy benchmarking, with EUROPEAN references (France, Germany, Spain) and in euros. The metrics of each axis, … - [GPU/LLM FinOps: frameworks, metrics and state of the art (tool by tool)](https://blog.lo0.es/en/posts/gpu-llm-finops-frameworks-metrics-state-of-the-art/index.md): A data introduction to FinOps tooling for GPU/LLM clusters: why the GPU is a different cost problem, the metrics, the three phases, OpenCost's allocation mechanics (node level, via … - [LLM inference benchmarking: frameworks, metrics and the state of the art (tool by tool)](https://blog.lo0.es/en/posts/llm-inference-benchmarking-frameworks-metrics-state-of-the-art/index.md): A data introduction to LLM inference benchmarking tooling: the metrics (TTFT, TPOT/ITL, throughput, goodput) and how they decompose, the difference between a single-process … - [Energy benchmarking in AI: frameworks, metrics and state of the art (tool by tool)](https://blog.lo0.es/en/posts/ai-energy-benchmarking-frameworks-metrics-state-of-the-art/index.md): A data-led introduction to energy and carbon measurement tooling in AI: the metrics (J/token, Wh, gCO2), the measurement layers (NVML/DCGM, RAPL, eBPF), the data sheets for Zeus, … - [Serving reasoning models: the invisible scratchpad that decides your latency and your bill](https://blog.lo0.es/en/posts/serving-reasoning-models-invisible-scratchpad-latency-bill/index.md): Serving o1/R1/Qwen3-style models in production: the reasoning block, the hidden tokens billed as output, the thinking budget (thinking_token_budget, budget forcing), the real … - [Long context and KV offloading: when the notebook does not fit on the desk](https://blog.lo0.es/en/posts/long-context-kv-offloading-notebook-desk/index.md): Long context (128k–1M tokens) is not a model problem, it is a memory problem: the KV cache explodes and does not fit in HBM. The KV memory hierarchy (HBM→DRAM→NVMe→network), when … - [Speeding up model cold start: from minutes to seconds](https://blog.lo0.es/en/posts/speeding-up-model-cold-start-minutes-to-seconds/index.md): The cold start of a model server is not one single thing: it is five line items, process init, CUDA context, allocator, weight loading disk→HBM, and CUDA graph capture + kernel … - [FinOps and multi-tenancy on the GPU cluster: who pays for what](https://blog.lo0.es/en/posts/finops-multi-tenancy-gpu-cluster-who-pays/index.md): A GPU cluster shared between teams has no monthly provider invoice: you have to manufacture the cost unit from scratch. This post derives the €/token from hardware depreciation and … - [Multimodal on-premise: serving a VLM with vLLM (vision + language)](https://blog.lo0.es/en/posts/multimodal-on-premise-serving-vlm-vllm/index.md): How to serve a vision-language model (the Qwen-VL family) with vLLM to understand documents and images, why an image costs thousands of tokens and when it is worth it against … - [Hardening and secrets in the sovereign LLM stack: defence in depth](https://blog.lo0.es/en/posts/hardening-secrets-sovereign-llm-stack-defence-in-depth/index.md): Layered hardening of a sovereign on-premise LLM stack: secrets out of git (sealed-secrets, SOPS/age, External Secrets + Vault), default-deny NetworkPolicy with egress control and … - [GitOps for the inference stack with Flux: operating the assistant as code](https://blog.lo0.es/en/posts/gitops-inference-stack-flux/index.md): A complete LLM assistant is six or more services across several namespaces. Deploying and patching it by hand does not scale and is not auditable. This post explains how to operate … - [Serving embeddings and rerankers with TEI in production](https://blog.lo0.es/en/posts/serving-embeddings-rerankers-tei-production/index.md): Text Embeddings Inference (TEI) is the engine of the RAG data plane: a Rust server that serves embeddings, rerankers and classification with token-based dynamic batching, an … - [End-to-end document ingestion: from PDF to indexed chunk](https://blog.lo0.es/en/posts/end-to-end-document-ingestion-pdf-to-indexed-chunk/index.md): The RAG ingestion pipeline stage by stage (parse, clean, chunk, enrich, embed, index) with layout-aware parsing (Docling), chunking strategies compared, deduplication, and the … - [Taking RAG to the CPU: separating the data plane from the generation plane](https://blog.lo0.es/en/posts/rag-cpu-data-plane-generation-plane/index.md): Almost all of RAG (chunking, embeddings, HNSW index, hybrid search, lightweight reranking) fits on CPU and is often where it belongs. Separating the data plane (CPU, … - [Serving several models on a single GPU: co-residency, model-swapping and sleep mode](https://blog.lo0.es/en/posts/serving-several-models-single-gpu-co-residency-swapping-sleep-mode/index.md): When you have more models than VRAM (an agent LLM, rerankers, an alternate model and the occasional big one) there are three ways to make them coexist on one GPU: co-reside them if … - [Sharing one GPU between several workloads: time-slicing, MPS and MIG](https://blog.lo0.es/en/posts/sharing-one-gpu-time-slicing-mps-mig/index.md): You have one GPU (or a few) and you want to run several workloads on it. There are three mechanisms for splitting it up: software time-slicing, spatial MPS and hardware MIG, and … - [Runbook: caging the AI agent — bubblewrap on the client, Tetragon on the cluster](https://blog.lo0.es/en/posts/runbook-caging-ai-agent-bubblewrap-tetragon/index.md): Operational procedure for isolating AI agents in both domains, with commands to copy and paste. On the client (workstation): install ai-jail + bubblewrap, the per-project .ai-jail … - [The contractor with the master key: isolating AI agents from workstation to cluster](https://blog.lo0.es/en/posts/isolating-ai-agents-workstation-to-cluster/index.md): An AI agent that writes and runs code is a contractor you open the door to: competent, well-meaning, but carrying a tool belt that could have been poisoned in a supply-chain … - [QLoRA runbook: from dataset to adapter served in multi-LoRA (operational procedure)](https://blog.lo0.es/en/posts/qlora-runbook-dataset-to-served-adapter/index.md): Step-by-step operational procedure to fine-tune an SLM with QLoRA on a single RTX 4090 (environment, dataset, TRL/PEFT script, monitoring, validation, adapter versioning) and serve … - [QLoRA and multi-LoRA at the limit on small models](https://blog.lo0.es/en/posts/qlora-multi-lora-limit-small-models/index.md): How QLoRA makes it possible to fine-tune a 3-8B SLM on a single RTX 4090 (NF4, double quantisation, paged optimizers) and why a base SLM + N low-rank adapters is the natural … - [Aggressive RAG in small models: trading parameters for retrieval](https://blog.lo0.es/en/posts/aggressive-rag-small-models-trading-parameters-for-retrieval/index.md): Why an SLM can compensate for its lower parametric knowledge by moving facts from the weights into the context via aggressive retrieval, and what the real tension with prefill cost … - [Aggressive quantisation (state of the art): from 4-bit to ternary](https://blog.lo0.es/en/posts/aggressive-quantisation-4-bit-to-ternary/index.md): Why naive post-hoc quantisation collapses below 4 bits, what AQLM, QuIP# and QTIP do at 2 bits, and why ternary BitNet b1.58 is native training, not compression. - [Test-time quantization: quantising on the fly with no calibration dataset](https://blog.lo0.es/en/posts/test-time-quantization-without-calibration-dataset/index.md): What test-time quantization (TTQ) is: deriving the quantisation scales from the real activations at inference time, with no offline pass and no calibration corpus, and why its … - [Device-native architectures: fine-grained MoE and pre-attention router](https://blog.lo0.es/en/posts/device-native-architectures-fine-grained-moe-pre-attention-router/index.md): Designing the model from scratch for the device's constraints instead of compressing a dense one built for cloud: fine-grained MoE, sparse FFN and a pre-attention router that … - [Self-speculative decoding: the model that gets ahead of itself](https://blog.lo0.es/en/posts/self-speculative-decoding-model-gets-ahead-of-itself/index.md): How a small model can act as its own draft by exiting at an intermediate layer (early-exit) and verifying itself with the full forward pass, at zero extra memory: LayerSkip, SWIFT, … - [The roofline flips: why optimising small models is a different performance game](https://blog.lo0.es/en/posts/roofline-flips-optimising-small-models/index.md): Why the roofline model changes regime in SLMs: the bottleneck stops being HBM and becomes the fixed per-step overhead, and almost every intuition from the large-model posts stops … - [The corridors and the security guard: PCIe topology, GPUDirect P2P and ACS](https://blog.lo0.es/en/posts/pcie-topology-gpudirect-p2p-acs-security-guard/index.md): Inside an inference node, the GPUs talk to each other over NVLink, but everything else (the weights coming off disk, the data arriving over the network, the KV exchanged between … - [The signature-dish specialist: vLLM's attention backend (FlashAttention, FlashInfer and the prefill/decode asymmetry)](https://blog.lo0.es/en/posts/vllm-attention-backend-flashattention-flashinfer-prefill-decode-asymmetry/index.md): Of all the kernels an LLM runs, almost all are standard matrix multiplications. Only one needs a specialist: attention. And it turns out it needs two different specialists, because … - [The pantry of pigeonholes: PagedAttention and the vLLM block manager](https://blog.lo0.es/en/posts/pagedattention-vllm-block-manager/index.md): The KV cache grows a little with every token, and reserving it contiguously, as the first servers did, wasted 60-80% of the HBM in fragmentation. PagedAttention applies to the KV … - [The pass: the maître d' who builds every round — vLLM's scheduler step](https://blog.lo0.es/en/posts/vllm-scheduler-step-maitre-d-each-round/index.md): On every iteration of the inference engine, someone decides which requests enter the GPU and how many tokens each one processes. That 'someone' is the scheduler, and its decision, … - [The chef who calls out every order: SMs, CUDA streams and CUDA graphs, or why the GPU gets bored generating tokens](https://blog.lo0.es/en/posts/sms-cuda-streams-cuda-graphs-inference/index.md): In decode, every token launches hundreds of tiny kernels, and each launch costs 5-10 µs of CPU. The kernels are so small that the GPU finishes them before the CPU calls out the … - [The pantry lift: from disk to HBM, or why the kitchen opens late](https://blog.lo0.es/en/posts/pantry-lift-from-disk-to-hbm-why-the-kitchen-opens-late/index.md): Before the first token you have to push 140 GB of weights from disk up to HBM. Hardly anyone times that journey, disk → page cache → host buffer → PCIe → HBM, and it is the one … - [The kitchen door the maître d' never looked at: network NUMA, Cilium eBPF and DRANET, the fourth leg of pinning](https://blog.lo0.es/en/posts/network-numa-cilium-ebpf-dranet-fourth-leg-pinning/index.md): The Topology Manager pins CPU, memory and GPU to the same NUMA node, but it never asks where the NIC is. On an inference node with the network at 200/400 Gb/s, that fourth leg, the … - [Choosing the switchboard: which OSS gateway to put in front, and why the licence decides before the features](https://blog.lo0.es/en/posts/choosing-oss-gateway-llm-inference/index.md): The gateway is the piece every request crosses and the most expensive one to rip out later: everything couples to it. Choosing it well starts with the licence (can you really own … - [The maître d' who only seats you if you all fit at one table: CPU, Memory and Topology Manager on RKE2](https://blog.lo0.es/en/posts/cpu-memory-topology-manager-rke2/index.md): Pinning NUMA by hand with numactl does not scale to a cluster. The kubelet automates it with three components, CPU Manager, Memory Manager and Topology Manager, which, under the … - [The floor next door: NUMA, hugepages and CPU isolation, or why your GPU waits on the kernel](https://blog.lo0.es/en/posts/numa-hugepages-cpu-isolation-gpu-waits-kernel/index.md): A 4×H100 node is a two-socket server: two NUMA nodes, each with its own cores, its own memory and half of the GPUs and NICs. If the host threads of inference land on the wrong … - [The shared bench: NVLink, NVSwitch and NCCL, the cable every token crosses in tensor parallel](https://blog.lo0.es/en/posts/nvlink-nvswitch-nccl-cable-every-token-crosses/index.md): Tensor parallelism does not split a model into four independent chunks: it forces the GPUs to add up partial results, an all-reduce, twice per layer, dozens of times per token. … - [Langfuse from the inside: the sorting centre that must not become the bottleneck it came to observe](https://blog.lo0.es/en/posts/langfuse-inside-sorting-centre-bottleneck/index.md): Langfuse v3 is not 'a trace database': it is six services (Web, Worker, Postgres, ClickHouse, Redis/Valkey, S3/Blob) joined by an asynchronous ingestion pipeline. This post takes … - [Pruning LLM models: cutting without amputating](https://blog.lo0.es/en/posts/llm-pruning-cutting-without-amputating/index.md): How to remove up to 50% of an LLM's weights without retraining it and without users noticing the difference. SparseGPT, Wanda, NVIDIA's 2:4 sparsity and what all of this means for … - [Prefix cache: hit rate engineering to go from 15% to 75%](https://blog.lo0.es/en/posts/prefix-cache-hit-rate-engineering/index.md): A 75% prefix cache hit rate is the same as multiplying your prefill capacity by 4 without adding hardware. Most deployments sit at 10-20% for avoidable reasons: timestamps in … - [Optimising prefill in vLLM: the knobs your TTFT will not forgive](https://blog.lo0.es/en/posts/optimising-prefill-vllm-knobs-ttft-will-not-forgive/index.md): Chunked prefill, prefix caching, FP8 KV cache and max-num-batched-tokens explained from scratch with real numbers. For engineers who want to squeeze an RTX 4090 or an L40 without … - [Optimising decode in vLLM: squeezing every token out of small hardware](https://blog.lo0.es/en/posts/optimising-decode-vllm-squeezing-every-token-small-hardware/index.md): PagedAttention, speculative decoding with EAGLE-3, quantised KV cache and gpu-memory-utilization tuning to get the most out of an RTX 4090 or an L40. With numbers. - [One big replica or many small ones: the decision that defines your platform](https://blog.lo0.es/en/posts/one-big-replica-or-many-small-ones-platform-decision/index.md): TP=4×1 replica vs TP=2×2 replicas on 4 GPUs: this is not just a technical decision, it defines fault tolerance, autoscaling granularity and the latency/throughput crossover point. … - [Knowledge Distillation: teaching a small model to think like a big one](https://blog.lo0.es/en/posts/knowledge-distillation-teaching-small-model-think-like-big-one/index.md): How to transfer the knowledge of a 405B model to an 8B one without the small model learning from scratch. The maths of distillation, the types (offline, online, on-policy), and why … - [Instrumenting vLLM with OTel: measuring what the optimisations really do](https://blog.lo0.es/en/posts/instrumenting-vllm-otel-measuring-optimisations/index.md): How to connect vLLM to an OTel pipeline to measure chunked prefill, prefix caching, speculative decoding, FP8 KV cache and concurrency precisely. Two signals, five critical … - [FP8 end to end: enable it, measure quality and decide with data](https://blog.lo0.es/en/posts/fp8-end-to-end-enable-measure-quality-decide/index.md): FP8 on an H100 is equivalent to getting 1.4 extra replicas for free: +40-60% throughput, ×2 VRAM for the KV cache. But switching it on without measuring the quality degradation is … - [Batch sizing in vLLM: the two-hour grid search worth weeks of hardware](https://blog.lo0.es/en/posts/vllm-batch-sizing-two-hour-grid-search/index.md): Most deployments leave 40-60% of their throughput on the table by using vLLM's defaults. A complete methodology for finding the optimal configuration of max-num-seqs and … - [Debezium and CDC: the notary who hears changes before anyone asks](https://blog.lo0.es/en/posts/debezium-cdc-notary-who-hears-changes/index.md): How Debezium uses the PostgreSQL WAL to capture every change in the database, DELETEs included, and propagate it to Kafka and Qdrant in under 100 ms, with no polling and no extra … - [Function calling and tool-augmented retrieval: the detective who knows which file to ask for](https://blog.lo0.es/en/posts/function-calling-tool-augmented-retrieval-detective-knows-which-file/index.md): How function calling turns an LLM into an agent that decides which source to consult: the ReAct pattern (Reasoning + Acting), the three heterogeneous data sources (vector store, … - [Evaluating a RAG system without fooling yourself: RAGAS, the golden dataset and the four metrics that matter](https://blog.lo0.es/en/posts/evaluating-rag-ragas-golden-dataset-metrics/index.md): RAGAS breaks the evaluation of a RAG system down into four orthogonal metrics, faithfulness, answer relevance, context precision and context recall, each pointing at a different … - [Semantic cache in RAG: the receptionist with a photographic memory](https://blog.lo0.es/en/posts/semantic-cache-rag-receptionist-photographic-memory/index.md): How a semantic cache cuts GPU compute in a production RAG pipeline by between 40% and 60%, intercepting semantically equivalent queries before they reach the retriever or the LLM. - [PostgreSQL + Qdrant in RAG ingestion: the postman who keeps two worlds in sync](https://blog.lo0.es/en/posts/postgresql-qdrant-rag-ingestion-postman-syncing-two-worlds/index.md): How to keep PostgreSQL and Qdrant in sync in a production RAG pipeline: outbox pattern, CDC with Debezium, microservice architecture and throughput maths. - [Ontologies and knowledge graphs in LLMOps: the Linnaean nomenclature that holds up the six pipeline stages](https://blog.lo0.es/en/posts/ontologies-knowledge-graphs-llmops-linnaean-nomenclature/index.md): Why an explicit ontology is the nomenclature without which the six LLMOps pipeline stages are speaking different languages, what TBox/ABox/RDF/OWL/SHACL/SKOS/JSON-LD really are in … - [Embeddings in 2026: the three families (dense, sparse, multi-vector), the model zoo and the decision that matters in production](https://blog.lo0.es/en/posts/embeddings-2026-dense-sparse-multivector-families/index.md): Why an embedder is always three different models at once (dense, sparse, multi-vector), what changes between them in quality, storage and latency, the 2026 open source model zoo … - [Mixed NVIDIA + Intel environments for LLM inference: from the central H100 cluster to the NUC in the branch office](https://blog.lo0.es/en/posts/mixed-nvidia-intel-environments-llm-inference/index.md): A serious production cluster in 2026 is no longer 'all NVIDIA' or 'all Intel', it is heterogeneous by design. NVIDIA H100 SXM handles the big LLM in the central DC; Intel Xeon 6 … - [Incident response runbooks for LLM inference: every alert to a concrete action with Kafka and Keep](https://blog.lo0.es/en/posts/incident-response-runbooks-llm-inference-kafka-keep/index.md): The DCGM and vLLM alerts documented in the previous posts are of no use without a codified procedure for each one. This post translates every anomaly into its concrete action … - [Anatomy of the twelve DCGM and five vLLM metrics: analogies, documented anomalies and real cases 2024-2026](https://blog.lo0.es/en/posts/anatomy-dcgm-vllm-metrics-analogies-documented-anomalies/index.md): The previous post on GPU observability listed the twelve DCGM metrics plus five vLLM ones with G/A/R thresholds. This one goes deeper: each metric with its own explanatory analogy, … - [The LLM inference router: the L7 switchboard we called a LoadBalancer in the canary post](https://blog.lo0.es/en/posts/llm-inference-router-l7-switchboard/index.md): In the previous post on canary, blue-green and shadow we generically called "LoadBalancer" the piece that splits traffic between the v1 and v2 pools. The description was accurate … - [Canary, blue-green and shadow for LLM models: how to deploy a new version without breaking the SLO](https://blog.lo0.es/en/posts/canary-blue-green-shadow-llm-models/index.md): Promoting a new LLM model to the production cluster without cutting traffic or breaking the SLO demands a progressive deployment strategy. The three options (blue-green, canary and … - [Autoscaling LLM inference on Kubernetes: HPA with custom metrics and KEDA for vLLM](https://blog.lo0.es/en/posts/llm-inference-autoscaling-kubernetes-hpa-keda/index.md): Classic Kubernetes autoscaling (HPA on CPU) does not work for LLM inference because the pod's CPU barely moves. There are four correct signals and they come from the engine: … - [GPU observability for LLM inference: the twelve DCGM and vLLM metrics that dictate the health of your production](https://blog.lo0.es/en/posts/gpu-observability-llm-inference-twelve-dcgm-vllm-metrics/index.md): A GPU is not observed with one metric. It is observed with a panel of twelve, spread across four families (compute, memory, thermal-power, health) and combined with five metrics … - [Capacity planning for on-premise LLM inference: how to size GPUs from an SLO](https://blog.lo0.es/en/posts/capacity-planning-on-premise-llm-inference/index.md): Before buying GPUs you need to know how many. This post takes apart the capacity planning calculation for an LLM inference cluster, starting from the SLO (TTFT P95, TPOT, requests … - [Technical controls: the ENS × ISO 42001 × EU AI Act cross-mapping over the on-premise LLM architecture](https://blog.lo0.es/en/posts/technical-controls-ens-iso-42001-eu-ai-act-cross-mapping/index.md): A cross-mapping of the technical controls of the Esquema Nacional de Seguridad (ENS, Spain's national security framework, RD 311/2022, 74 measures) against the 38 Annex A controls … - [ISO/IEC 42001: the operations manual for the AI system — how the AIMS fits over the on-premise LLM platform described in this blog](https://blog.lo0.es/en/posts/iso-42001-aims-on-premise-llm-platform/index.md): ISO/IEC 42001:2023 is the first certifiable AI management standard: 7 Annex SL clauses + 38 Annex A controls. This post takes it apart functionally and maps it control by control … - [EU AI Act: the technical dossier, article by article, over the blog's on-premise LLM architecture](https://blog.lo0.es/en/posts/eu-ai-act-technical-dossier-on-premise-llm-architecture/index.md): An article-by-article mapping of Regulation (EU) 2024/1689 (the EU AI Act) onto the on-premise LLM architecture this blog has described. For each relevant article (5 prohibitions, … - [LLM Guard: the sworn translator with a notebook of equivalences — anatomy, scanners and its integration with Langfuse, vLLM and LiteLLM](https://blog.lo0.es/en/posts/llm-guard-anatomy-scanners-langfuse-vllm-litellm/index.md): Functional deep dive into LLM Guard (Protect AI): the piece inside the guardrail layer that stands out for its Anonymize → LLM → Deanonymize pattern with a central Vault, its … - [Guardrails and safety in LLMs: the four lines of defence of a production request](https://blog.lo0.es/en/posts/llm-guardrails-safety-four-lines-of-defence/index.md): Why guardrails are the only safety layer that lives on the request path, what separates them from offline evals, and how the four lines of defence (input, retrieval, tool, output) … - [Seven greenfield deployment phases for an on-premise LLM platform: from hardware in the room to the first production token](https://blog.lo0.es/en/posts/greenfield-deployment-phases-on-premise-llm-platform/index.md): Third post of the architectural trilogy. The two previous ones settled what gets built and why; this one settles when. Seven nominal phases (F0–F6) that take an on-premise LLM … - [Five maturity levels of the platform under the LLM: from a Linux server to a cluster ready for vLLM](https://blog.lo0.es/en/posts/five-maturity-levels-platform-under-llm/index.md): The previous post assumed seven layers were ready; this one walks through what sits beneath. From a server with Linux to a cluster prepared to run vLLM, there are five maturity … - [Structured output: the form with dropdowns that strikes out invalid answers before the model picks one — Outlines, XGrammar, LLGuidance and the maths of the bitmask](https://blog.lo0.es/en/posts/structured-output-bitmask-outlines-xgrammar-llguidance/index.md): How you guarantee 100 % that an LLM produces parseable JSON conforming to a schema: the principle of the logits bitmask, the automata (FSM/PDA) that decide which tokens are valid … - [Continuous batching: the salon with 8 chairs that does not wait for the slow customer — Orca, vLLM, chunked prefill and goodput](https://blog.lo0.es/en/posts/continuous-batching-orca-vllm-chunked-prefill-goodput/index.md): Why static batching left the GPU at 20-40 % utilisation, what Orca (OSDI '22) changes with iteration-level scheduling, how vLLM materialises it with PagedAttention, what chunked … - [Anatomy of an on-premise LLM inference stack: the seven layers that have to hold each other up](https://blog.lo0.es/en/posts/anatomy-on-premise-llm-inference-stack-seven-layers/index.md): The on-premise LLM inference stack is not a model server: it is seven layers (inference, embeddings, vector store, gateway, observability, control plane, dependency tracking) whose … - [Multi-LoRA serving: the single translator with a thousand glossaries — shared base, thousands of concurrent adapters and the SGMV kernel](https://blog.lo0.es/en/posts/multi-lora-serving-shared-base-concurrent-adapters-sgmv-kernel/index.md): How hundreds or thousands of concurrent LoRA adapters are served against a single base without replicating the base, what the SGMV kernel (Segmented Gather Matrix-Vector … - [MoE inference: the call centre with 256 specialists and 8 taking each call — fundamentals, expert parallel and the economics of DeepSeek-V3](https://blog.lo0.es/en/posts/moe-inference-fundamentals-expert-parallel-deepseek-economics/index.md): Why a MoE like DeepSeek-V3 has 671B total parameters but activates only 37B per token, what changes against dense in throughput and memory, how 256 experts are spread across 32 or … - [FlashAttention v1/v2/v3/v4: the librarian who never clears the desk — IO-awareness, async and Blackwell's asymmetry](https://blog.lo0.es/en/posts/flashattention-librarian-io-awareness-async-blackwell-asymmetry/index.md): Why attention was memory-bound on modern GPUs and how FlashAttention turns it around with tiling and online softmax; the three ideas in FA2 (sequence parallelism, work … - [Speculative decoding: the secretary who types ahead of the boss — fundamentals, maths and state of play in May 2026](https://blog.lo0.es/en/posts/speculative-decoding-fundamentals-maths-state-of-play/index.md): How speculative decoding works exactly (draft + verify + rejection sampling), why the maths guarantees quality does not degrade, the 1/(1-α) algorithmic ceiling, the five dominant … - [Quantization for LLM inference: FP8, INT4 (GPTQ, AWQ) and GGUF — the model's accounting zoom](https://blog.lo0.es/en/posts/quantization-llm-inference-fp8-int4-gguf/index.md): Why a 70B model fits in 24 GB once quantised, what the maths behind it is (scale + zero-point), the practical differences between PTQ and QAT, the four dominant formats in 2026 … - [LLM tracing with OpenTelemetry GenAI: the aircraft black box the field stabilised in 2026](https://blog.lo0.es/en/posts/llm-tracing-opentelemetry-genai/index.md): Why LLM observability is not being reinvented by proprietary vendors, what OpenTelemetry's `gen_ai.*` convention demands, how the SDK → Collector → Langfuse + Tempo + Prometheus … - [LLM-as-judge: the exam marker who grades other models without turning into an oracle](https://blog.lo0.es/en/posts/llm-as-judge-exam-marker-not-oracle/index.md): Why using one LLM to evaluate another works, the three canonical patterns in 2026 (G-Eval, Prometheus 2, panel of judges), the maths of calibration with Cohen's kappa, the four … - [Modern alignment: DPO, KTO, ORPO and SimPO — the sommelier who learns without a reward model](https://blog.lo0.es/en/posts/modern-alignment-dpo-kto-orpo-simpo/index.md): Why classic RLHF is practically extinct in 2026, what happened to the reward model, how DPO replaced it with a change of variable, and where KTO (binary signal), ORPO (SFT + … - [Reranker and hybrid retrieval: the committee that decides the 5 chunks the LLM will actually read](https://blog.lo0.es/en/posts/reranker-hybrid-retrieval-five-chunks-llm-reads/index.md): Why dense retrieval on its own is not enough for a RAG in production, how it combines with BM25 / SPLADE in hybrid retrieval, what a cross-encoder reranker adds, late interaction … - [RAG corpus curation: the active librarian who decides what goes in, what goes out and what gets signed](https://blog.lo0.es/en/posts/rag-corpus-curation-active-librarian/index.md): Why a RAG system without active corpus curation is a system that learns to get things wrong in production. The five curation layers (schema validation, exact + near + semantic … - [Evals for LLMs: the layer after tracing that decides whether your model performs or only looks like it does](https://blog.lo0.es/en/posts/evals-llm-layer-after-tracing/index.md): Why tracing is not evaluation, what a golden dataset really is, how to build an LLM-as-judge that does not lie, the mathematics that separate a real eval from a decorative … - [Data versioning for LLMOps: DVC, lakeFS and the reproducible golden dataset challenge](https://blog.lo0.es/en/posts/data-versioning-llmops-dvc-lakefs-reproducible-golden-dataset/index.md): Why Git is not enough to version the datasets that feed an LLM system, the four artefacts worth versioning (training, RAG corpus, golden eval, enriched retrain), the DVC + lakeFS … - [Retrain: closing the loop between the production incident and the adapter that fixes it](https://blog.lo0.es/en/posts/retrain-closing-the-loop-feedback-dataset-adapter/index.md): The most neglected stage of the LLMOps pipeline. How feedback is captured (explicit + implicit), triaged by root cause, how the dataset is enriched, how cadence is decided … - [Prompt versioning: the contract that stops a five-word change sinking your system](https://blog.lo0.es/en/posts/prompt-versioning-contract-five-word-change/index.md): Why the prompt is code and needs real versioning. A pattern of three primitives (version, label, cache), the two dominant tools in 2026 (Langfuse and MLflow Prompts), integration … - [Disaggregated serving: prefill and decode in specialised pods](https://blog.lo0.es/en/posts/disaggregated-serving-prefill-decode-specialised-pods/index.md): Why prefill and decode have incompatible GPU profiles, how they are split into specialised pods with KV cache transfer over NIXL/RDMA, and what real numbers the pattern delivers in … - [Continuous fine-tuning in production: from real traffic to the deployed adapter](https://blog.lo0.es/en/posts/continuous-fine-tuning-production-real-traffic-deployed-adapter/index.md): How an on-premise continuous fine-tuning cycle is built in 2026: alignment techniques by type of signal, dataset versioning on Postgres, eval gates as SQL predicates, A/B testing … - [The six-stage LLMOps pipeline: global architecture and a deep dive into every component](https://blog.lo0.es/en/posts/six-stage-llmops-pipeline-architecture-and-deep-dive/index.md): The third post in the MLOps for LLMs series. Global architecture of a modern LLMOps system with a reusable master SVG, and a deep dive into each of the six stages (Data, Tune, … - [MLOps specific to LLMs in 2026: the landscape of three modalities, six stages and ten tools that make them work](https://blog.lo0.es/en/posts/mlops-for-llms-landscape-2026/index.md): Why LLMOps is structurally different from traditional MLOps, the end-to-end six-stage pipeline (data → tune → eval → deploy → observe → retrain), the three modalities of preparing … - [KV cache: the working memory that holds up LLM inference](https://blog.lo0.es/en/posts/kv-cache-working-memory-llm-inference/index.md): What the KV cache is, why LLM inference would be unworkable without it, how much VRAM it eats and how it dictates what fits on an RTX 4090 or on a cluster of H100s. - [Welcome to the lo0 blog](https://blog.lo0.es/en/posts/welcome-lo0-blog/index.md): First post of the lo0.es technical blog ## Fundamentos - [Real ClickHouse capacity in Langfuse v4: 2,782 bytes per observation, and why almost half of that is not data](https://blog.lo0.es/en/posts/clickhouse-capacity-langfuse-v4-bytes-per-observation/index.md): Sixth article in the series on operating self-hosted Langfuse v4. Langfuse publishes no capacity figure at all: not bytes per observation, not events per second, not the expected … - [deepagents on a cluster of your own: the SDK is free, the server is not, and what you have to build in between](https://blog.lo0.es/en/posts/deepagents-free-sdk-licensed-server/index.md): Opening of the agentic platform vertical. deepagents is LangChain's agent harness, and the question that matters when you put it into a platform of your own is not what it can do, … - [Making the model prefer our tools: the priority that exists in no layer, the two primitives that do, and the `_meta` that crosses the gateway](https://blog.lo0.es/en/posts/mcp-tool-priority-visibility-and-exclusion/index.md): A follow-up to the article on the MCP gateway. When an agent has native tools, a gateway of its own and three third-party servers, the operational question is how you get the model … ## Kubernetes - [Hubble: eBPF network observability, state of the art 2026 and the new frontier with AI agents](https://blog.lo0.es/en/posts/hubble-ebpf-network-observability-ai-agents/index.md): Hubble as a modern tcpdump with native Kubernetes context: distributed architecture, L3-L7 flow logs, Prometheus metrics, UI, use cases. State of the art 2026 (Cilium 1.19, IP … - [Tetragon: Cilium's security cousin that sees every syscall in the kernel](https://blog.lo0.es/en/posts/tetragon-cilium-security-syscalls-kernel/index.md): An extensive introduction to Tetragon, the eBPF-based runtime security and observability engine: architecture, TracingPolicy / TracingPolicyNamespaced, hook points (kprobes, … - [eBPF from zero to Cilium: how the kernel learned to skip its own TCP/IP stack](https://blog.lo0.es/en/posts/ebpf-zero-to-cilium-kernel-tcp-ip-bypass/index.md): Basic eBPF (what it is, what it is not), eBPF in networking (XDP, TC, sock_ops), how Cilium skips the traditional TCP/IP stack and the CRDs of BGP Control Plane v2 … - [Kubernetes with Cilium BGP: services reachable without Ingress](https://blog.lo0.es/en/posts/kubernetes-cilium-bgp-services-without-ingress/index.md): How to configure Cilium BGP on RKE2 to expose ClusterIPs directly on the LAN ## Llmops pipeline - [The OSS catalogue for LLMOps in six stages: tool by tool, what each one does and when to choose it](https://blog.lo0.es/en/posts/oss-catalogue-llmops-six-stages-tool-by-tool/index.md): Third post in the series on the LLMOps pipeline, a companion to the forensic post and to the parallel OSS vs hyperscalers catalogue. This one focuses on the OSS side: for each of … - [The parallel catalogue: the six LLMOps stages in open source and on the hyperscalers (AWS, GCP, Azure)](https://blog.lo0.es/en/posts/parallel-catalogue-llmops-stages-open-source-hyperscalers/index.md): For each of the six stages of the LLMOps pipeline (Data, Tune, Eval, Deploy, Observe, Retrain) and the two cross-cutting components (prompt versioning, data versioning) we walked … - [Anatomy of an LLM request in production, May 2026: a tour of the six stages following a single request](https://blog.lo0.es/en/posts/anatomy-llm-request-production-tour-six-stages/index.md): The synthesis post of the blog. We rewind a single real request from the frame in which the user clicks back to the data that trained it 90 days earlier, and then follow it forward … ## Networking - [EVPN-VXLAN with Huawei CloudEngine: a practical guide](https://blog.lo0.es/en/posts/evpn-vxlan-huawei-cloudengine-practical-guide/index.md): Configuring an EVPN-VXLAN fabric with an ISIS underlay and BGP ECMP on Huawei CE6863 switches ## Physical infrastructure - [Cooling the AI datacenter (1/4): the thermal challenge](https://blog.lo0.es/en/posts/ai-datacenter-cooling-thermal-challenge/index.md): First article in the cooling series: why AI density has broken the air-cooling model. The spectrum of solutions from air to liquid and immersion, the thermal metrics (PUE, WUE, … - [Cooling the AI datacenter (2/4): air and its limits](https://blog.lo0.es/en/posts/ai-datacenter-cooling-air-and-its-limits/index.md): Second article in the series: air cooling. CRAC and CRAH, the geometry of airflow under the raised floor, the fan cube law, aisle containment, the ASHRAE thermal envelope and class … - [Cooling the AI datacenter (3/4): direct liquid to chip](https://blog.lo0.es/en/posts/ai-datacenter-cooling-direct-liquid-to-chip/index.md): Third article in the series: direct-to-chip liquid cooling. Inside the cold plate, how much heat the fluid captures, the TCS and FWS loops, the CDU and its approach temperature, … - [Cooling the AI datacenter (4/4): immersion and the megawatt horizon](https://blog.lo0.es/en/posts/ai-datacenter-cooling-immersion-megawatt-horizon/index.md): Fourth and final article in the series: single-phase and two-phase immersion, what has to change in a submerged server, the PFAS regulatory wall, why direct liquid to chip is … - [Datacenter physical infrastructure (1/4): the power chain](https://blog.lo0.es/en/posts/datacenter-physical-infrastructure-power-chain/index.md): First article in the series on physical infrastructure: the complete electrical chain of a datacenter, from the utility feed to the rack. N/N+1/2N redundancy, Tier levels, kW … - [Datacenter physical infrastructure (2/4): the generators](https://blog.lo0.es/en/posts/datacenter-physical-infrastructure-generators/index.md): Second article in the series: the generator sets. Starting in 10 seconds (NFPA 110), the ISO 8528 ratings and 'Data Center Continuous', sizing and transient response, diesel versus … - [Datacenter physical infrastructure (3/4): the switching](https://blog.lo0.es/en/posts/datacenter-physical-infrastructure-switching/index.md): Third article in the series: switching between sources. ATS versus STS, open and closed transition, the tolerance window of server power supplies, the NEC and UL standards, the A/B … - [Datacenter physical infrastructure (4/4): the UPS and the ride-through](https://blog.lo0.es/en/posts/datacenter-physical-infrastructure-ups-and-ride-through/index.md): Fourth and final article in the series: the UPS. The ride-through that covers generator start-up, the IEC 62040-3 topologies, eco-mode, the flywheel, lithium batteries against …