Token Pooling-Aware Training for Multi-Vector Embedding Models
Comments
Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules
Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules
Summary
Token pooling shrinks multi-vector retrieval indexes by merging similar token vectors, but aggressive compression can discard useful information. The author proposes pooling-aware training, which adds a retrieval loss computed on pooled document vectors alongside the normal unpooled loss, so the encoder learns to preserve what matters after merging.