The Inference Stack Behind OCRia

Inference-optimization write-up for OCRia-1, a document parsing model, reports a jump from about 285 to about 2,800 tokens per second on a single B200 GPU. The gains come from workload-aware mixed-precision quantization, speculative decoding with DFlash2, and a custom CUDA kernel for GDN state updates that cut per-layer time roughly…