microsoft/onnxruntime plugin-ep-cuda/v0.4.0
Comments
Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules
Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules
Summary
The microsoft/onnxruntime plugin-ep-cuda v0.4.0 release, published by Microsoft's ONNX Runtime project, adds CPU/GPU hybrid expert offloading for Mixture-of-Experts models that exceed GPU memory, plus new quantized MoE kernels (INT2/INT3/INT4, offline NVFP4) and long-context and speculative-decoding speedups.