Langfuse v4, day 2 (5 of 10): the worker's thirty-nine queues, and the three that decide whether you lose dataSep 14, 2026
Langfuse v4, day 2 (4 of 10): migrating from 3 to 4 with no window, and where the point of no return sitsSep 14, 2026
Langfuse v4, day 2 (3 of 10): two well-known agents instrumented, Open Deep Research and GPT ResearcherSep 14, 2026
Langfuse v4, day 2 (2 of 10): putting LangGraph in front, and what one agent turn costs in observationsSep 13, 2026
Completing Keycloak for MCP: the protected resource no SDK hands you ready-made, and the enterprise extension that skips the whole flowSep 12, 2026
Keycloak in an AI platform: where it fits, what it solves and the two standards MCP requires and it does not implementSep 12, 2026
Langfuse v4, day 2 (1 of 10): the data model changed underneath, and what really goes into a traceSep 12, 2026
The gateway does not live alone: the JWT that does not validate the audience, the user who never reaches the trace, and the seams you have to write by handSep 12, 2026
Sizing for agents: 275 input tokens for every output token, and why the bottleneck is still decodeSep 12, 2026
LiteLLM's MCP gateway: the second front door, the tool catalogue nobody bills, and why filtering it can make selection worseSep 11, 2026
Humans and agents on the same gateway: why `priority` does not prioritise, the 429 that brings down the whole pool, and the two ways to separate trafficSep 10, 2026
Virtual keys, budgets and limits in LiteLLM: the layer that decides who gets the GPU, and the four things the documentation gets wrongSep 8, 2026
LiteLLM on day 2: one worker per pod, six thousand seconds of timeout and the other five things to change before opening the trafficSep 8, 2026
LiteLLM and Langfuse: the operational pair, and the four places where the trace is lost between the gateway and the dashboardSep 7, 2026
Knative and scale-to-zero for LLM inference: when switching off the GPU saves money and when it costs you the SLOAug 31, 2026
KubeVela and Score: handing over the platform by CLI without teaching the developer KubernetesAug 31, 2026
Kubeflow in depth: which pieces are worth it in an on-premise LLM platform and which ones you already haveAug 31, 2026
Backstage as a self-service portal: giving your teams the LLM platform without giving them the clusterAug 31, 2026
Measuring the power of a GPU: NVML, DCGM and the sampling errors that invalidate your wattsAug 29, 2026
Model chain of trust (1/4): KServe and the Open Inference Protocol, the control plane your serving is missingJul 26, 2026
Model chain of trust (2/4): where the bytes live — registry, OCI artefacts and distribution to the GPU nodeJul 26, 2026
Volcano and Kueue: gang scheduling, queues and GPU quotas for distributed workloads on KubernetesJun 16, 2026
The reproducible harness: measuring cost, performance and energy in a single auditable experimentJun 16, 2026
From the SLO to the number of GPUs: how to size and justify the investment in inference hardwareJun 16, 2026
Energy in TCO and in regulation: cost, PUE, CSRD and the new European reporting obligation for datacentersJun 16, 2026
Measurement bias and reproducibility: why two benchmarks of the same model give figures that differ by up to 7×Jun 16, 2026
Full TCO of an on-premise GPU cluster: from capex to the all-in €/GPU-hour and the break-even against cloudJun 16, 2026
Cloud GPU: on-demand, reserved and spot, and why neoclouds change the board (with 2026 prices)Jun 16, 2026
LLM quality benchmarks: the contamination trap and the OSS tools for not fooling yourselfJun 16, 2026
Sovereign on-premise vs hyperscalers: the case with data (cost, energy, performance and sovereignty)Jun 14, 2026
Measuring energy in production: Kepler, DCGM and the practical stack (accuracy and overhead)Jun 14, 2026
Energy per token in Spain: methodology, the electricity market and the cases of speculationJun 14, 2026
Kubecost vs OpenCost vs the alternatives: what the commercial layer adds and when it is worth paying forJun 14, 2026
Cost, performance and energy: the three axes of LLM inference and the identity that ties them togetherJun 13, 2026
Serving reasoning models: the invisible scratchpad that decides your latency and your billJun 12, 2026
The signature-dish specialist: vLLM's attention backend (FlashAttention, FlashInfer and the prefill/decode asymmetry)Jun 8, 2026
The chef who calls out every order: SMs, CUDA streams and CUDA graphs, or why the GPU gets bored generating tokensJun 7, 2026
The kitchen door the maître d' never looked at: network NUMA, Cilium eBPF and DRANET, the fourth leg of pinningJun 6, 2026
Choosing the switchboard: which OSS gateway to put in front, and why the licence decides before the featuresJun 6, 2026
The maître d' who only seats you if you all fit at one table: CPU, Memory and Topology Manager on RKE2Jun 6, 2026
The floor next door: NUMA, hugepages and CPU isolation, or why your GPU waits on the kernelJun 6, 2026
The shared bench: NVLink, NVSwitch and NCCL, the cable every token crosses in tensor parallelJun 6, 2026
Langfuse from the inside: the sorting centre that must not become the bottleneck it came to observeJun 6, 2026
Function calling and tool-augmented retrieval: the detective who knows which file to ask forJun 4, 2026
Evaluating a RAG system without fooling yourself: RAGAS, the golden dataset and the four metrics that matterJun 4, 2026
Ontologies and knowledge graphs in LLMOps: the Linnaean nomenclature that holds up the six pipeline stagesJun 3, 2026
Embeddings in 2026: the three families (dense, sparse, multi-vector), the model zoo and the decision that matters in productionJun 3, 2026
Mixed NVIDIA + Intel environments for LLM inference: from the central H100 cluster to the NUC in the branch officeJun 2, 2026
Incident response runbooks for LLM inference: every alert to a concrete action with Kafka and KeepJun 2, 2026
Anatomy of the twelve DCGM and five vLLM metrics: analogies, documented anomalies and real cases 2024-2026Jun 2, 2026
Canary, blue-green and shadow for LLM models: how to deploy a new version without breaking the SLOJun 1, 2026
GPU observability for LLM inference: the twelve DCGM and vLLM metrics that dictate the health of your productionJun 1, 2026
Technical controls: the ENS × ISO 42001 × EU AI Act cross-mapping over the on-premise LLM architectureJun 1, 2026
ISO/IEC 42001: the operations manual for the AI system — how the AIMS fits over the on-premise LLM platform described in this blogJun 1, 2026
EU AI Act: the technical dossier, article by article, over the blog's on-premise LLM architectureJun 1, 2026
LLM Guard: the sworn translator with a notebook of equivalences — anatomy, scanners and its integration with Langfuse, vLLM and LiteLLMJun 1, 2026
Seven greenfield deployment phases for an on-premise LLM platform: from hardware in the room to the first production tokenMay 31, 2026
Five maturity levels of the platform under the LLM: from a Linux server to a cluster ready for vLLMMay 31, 2026
Structured output: the form with dropdowns that strikes out invalid answers before the model picks one — Outlines, XGrammar, LLGuidance and the maths of the bitmaskMay 30, 2026
Continuous batching: the salon with 8 chairs that does not wait for the slow customer — Orca, vLLM, chunked prefill and goodputMay 30, 2026
Anatomy of an on-premise LLM inference stack: the seven layers that have to hold each other upMay 30, 2026
Multi-LoRA serving: the single translator with a thousand glossaries — shared base, thousands of concurrent adapters and the SGMV kernelMay 30, 2026
MoE inference: the call centre with 256 specialists and 8 taking each call — fundamentals, expert parallel and the economics of DeepSeek-V3May 30, 2026
FlashAttention v1/v2/v3/v4: the librarian who never clears the desk — IO-awareness, async and Blackwell's asymmetryMay 29, 2026
Speculative decoding: the secretary who types ahead of the boss — fundamentals, maths and state of play in May 2026May 29, 2026
Quantization for LLM inference: FP8, INT4 (GPTQ, AWQ) and GGUF — the model's accounting zoomMay 27, 2026
LLM tracing with OpenTelemetry GenAI: the aircraft black box the field stabilised in 2026May 27, 2026
Modern alignment: DPO, KTO, ORPO and SimPO — the sommelier who learns without a reward modelMay 27, 2026
Reranker and hybrid retrieval: the committee that decides the 5 chunks the LLM will actually readMay 25, 2026
RAG corpus curation: the active librarian who decides what goes in, what goes out and what gets signedMay 25, 2026
Evals for LLMs: the layer after tracing that decides whether your model performs or only looks like it doesMay 25, 2026
MLOps specific to LLMs in 2026: the landscape of three modalities, six stages and ten tools that make them workMay 21, 2026