Scaling Postgres to 1M durable workflows per second
Comments
Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules
Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules
Summary
DBOS reports scaling its Postgres-backed durable workflow engine from 20K workflows/sec on one server to 1M on 50 RDS shards, using the PgDog sharding router. Past roughly 8-16 shards, dequeueing hit a feedback loop: slow shards attracted more concurrent pollers, and lock contention made them slower. Per-shard advisory-lock limits fixed it. The figures come from DBOS's own benchmark of no-op workflows with no steps, and the claim check could not independently confirm them. The code is open-source.