TParquet – Parquet for time-series, 50% compression with Pcodec
Comments
Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules
Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules
Summary
tparquet is an open-source extension to Parquet that adds a PCODEC compression codec for numeric columns, built on the pcodec library. On 151.5M rows of trade, quote, and order-book data from four market data providers, its files came out at 0.50× the size of ZSTD(9) Parquet, and every value reads…