Sizing for agents: 275 input tokens for every output token, and why the bottleneck is still decodeSep 12, 2026
Humans and agents on the same gateway: why `priority` does not prioritise, the 429 that brings down the whole pool, and the two ways to separate trafficSep 10, 2026
LiteLLM on day 2: one worker per pod, six thousand seconds of timeout and the other five things to change before opening the trafficSep 8, 2026
LiteLLM and Langfuse: the operational pair, and the four places where the trace is lost between the gateway and the dashboardSep 7, 2026
Model chain of trust (1/4): KServe and the Open Inference Protocol, the control plane your serving is missingJul 26, 2026
Serving reasoning models: the invisible scratchpad that decides your latency and your billJun 12, 2026
The signature-dish specialist: vLLM's attention backend (FlashAttention, FlashInfer and the prefill/decode asymmetry)Jun 8, 2026
The chef who calls out every order: SMs, CUDA streams and CUDA graphs, or why the GPU gets bored generating tokensJun 7, 2026
The floor next door: NUMA, hugepages and CPU isolation, or why your GPU waits on the kernelJun 6, 2026
The shared bench: NVLink, NVSwitch and NCCL, the cable every token crosses in tensor parallelJun 6, 2026
Function calling and tool-augmented retrieval: the detective who knows which file to ask forJun 4, 2026
Embeddings in 2026: the three families (dense, sparse, multi-vector), the model zoo and the decision that matters in productionJun 3, 2026
Anatomy of the twelve DCGM and five vLLM metrics: analogies, documented anomalies and real cases 2024-2026Jun 2, 2026
Canary, blue-green and shadow for LLM models: how to deploy a new version without breaking the SLOJun 1, 2026
GPU observability for LLM inference: the twelve DCGM and vLLM metrics that dictate the health of your productionJun 1, 2026
LLM Guard: the sworn translator with a notebook of equivalences — anatomy, scanners and its integration with Langfuse, vLLM and LiteLLMJun 1, 2026
Structured output: the form with dropdowns that strikes out invalid answers before the model picks one — Outlines, XGrammar, LLGuidance and the maths of the bitmaskMay 30, 2026
Continuous batching: the salon with 8 chairs that does not wait for the slow customer — Orca, vLLM, chunked prefill and goodputMay 30, 2026
Anatomy of an on-premise LLM inference stack: the seven layers that have to hold each other upMay 30, 2026
Multi-LoRA serving: the single translator with a thousand glossaries — shared base, thousands of concurrent adapters and the SGMV kernelMay 30, 2026
MoE inference: the call centre with 256 specialists and 8 taking each call — fundamentals, expert parallel and the economics of DeepSeek-V3May 30, 2026
FlashAttention v1/v2/v3/v4: the librarian who never clears the desk — IO-awareness, async and Blackwell's asymmetryMay 29, 2026
Speculative decoding: the secretary who types ahead of the boss — fundamentals, maths and state of play in May 2026May 29, 2026
Quantization for LLM inference: FP8, INT4 (GPTQ, AWQ) and GGUF — the model's accounting zoomMay 27, 2026
The OSS catalogue for LLMOps in six stages: tool by tool, what each one does and when to choose itMay 23, 2026
eBPF in local inference and statistical drift detection: closing the LLM observability loop in 2026May 20, 2026
LLM inference operators on Kubernetes: OME, vLLM Production Stack, NVIDIA Dynamo and llm-dMay 18, 2026