The story’s score and claims

Comments on No. 76 of 122 · GitHub · edition of Tuesday 17 November 2026

Benchmarking OSS inference vendors on cost/latency

Summary

The powerset-co inference-bench repo is a beta benchmark that compares open-source LLM inference providers on speed and list-price cost while serving a single AI research agent. It replays recorded FRAMES-based Wikipedia research conversations across providers, timing only model calls, with quality deliberately unmeasured.

Comments

Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules