An Open Field Guide to Inference Engineering

Modelplane, the open-source project from Upbound, published a field guide to inference engineering aimed at platform engineers who run open models themselves. It covers GPU memory sizing (weights, KV cache, runtime overhead), prefill versus decode bottlenecks, and when to adopt each optimization. Upbound's own agent traffic had a 59K-token median prompt and about two-thirds prefix-cache hits. The guide cites an 80% cost cut at high volume, plus Orca's and PagedAttention's gains, which the claim check could not verify.