Continuous batching: the salon with 8 chairs that does not wait for the slow customer — Orca, vLLM, chunked prefill and goodputMay 30, 2026
Multi-LoRA serving: the single translator with a thousand glossaries — shared base, thousands of concurrent adapters and the SGMV kernelMay 30, 2026
Speculative decoding: the secretary who types ahead of the boss — fundamentals, maths and state of play in May 2026May 29, 2026