## The KV Cache Tax: Why Inference Servers Run Out of Memory Before Compute A VRAM budget formula for LLM serving, and three optimization strategies mapped to the traffic patterns that trigger the OOM. ## The Model Fits, the Requests Don't VRAM used to be a training-time worry. You sized your cluster for the weights, the optimizer states, and the gradients, and once the model was trained, the memory math felt settled. Serving looked cheap by comparison: load the weights, run forward passes, done. Then you put the model behind real traffic, and it fell over at a load that made no sense. I was…