Length Is Cheap, Competition Is Not: Attention Predicts Context Rot in Retrieval

A preprint by Yinuo Liu (UC San Diego) argues that long-context retrieval failures come mainly from competition for attention rather than length. Across 126,473 trials on five open models of 0.5B to 3B parameters, neutral filler up to 32K tokens barely hurt accuracy, while statements mimicking the evidence format, especially…