From 22 GB to 5.3 GB: all of Hacker News, searchable offline
Comments
Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules
Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules
Summary
An open-source project compresses all 49.3 million Hacker News items from 2006 to 2026 into a 5.34 GB file searchable offline on a laptop CPU. It embeds each item once using EmbeddingGemma 2 on an integrated GPU, stores 256-bit binary codes, and reranks candidates at query time.