-
AITwo Workloads in a Trench Coat: Why Prefill and Decode Change Everything
LLM inference is two opposite workloads — compute-bound prefill and memory-bandwidth-bound decode. Here's how continuous batching, PagedAttention, and disaggregation evolved to deal with it.
#llm-inference #prefill-decode #vllm #pageattention #continuous-batching #gpu-architecture #nvidia-dynamo #kv-cache #ai-engineering -
AIServing the Machine: How LLM Inference Runs at Planetary Scale
From PagedAttention to GB200 racks, from token economics to MCP — a systems engineer's tour of how frontier models serve millions of users simultaneously.
#llm-inference #gpu-architecture #vllm #pageattention #continuous-batching #mcp #api-design #quantization #speculative-decoding #ai-engineering