Serving reasoning models: the invisible scratchpad that decides your latency and your billJun 12, 2026
Structured output: the form with dropdowns that strikes out invalid answers before the model picks one — Outlines, XGrammar, LLGuidance and the maths of the bitmaskMay 30, 2026
Continuous batching: the salon with 8 chairs that does not wait for the slow customer — Orca, vLLM, chunked prefill and goodputMay 30, 2026
MoE inference: the call centre with 256 specialists and 8 taking each call — fundamentals, expert parallel and the economics of DeepSeek-V3May 30, 2026
Speculative decoding: the secretary who types ahead of the boss — fundamentals, maths and state of play in May 2026May 29, 2026
LLM inference operators on Kubernetes: OME, vLLM Production Stack, NVIDIA Dynamo and llm-dMay 18, 2026