Attention magnitude is unrelated to a token's causal contribution to the answer in KV cache eviction

Using a controlled leave-one-out probe, we find that attention magnitude is unrelated to a token's causal contribution to the answer (Spearman $ρ=-0.004$), challenging the premise behind dominant eviction methods.

http://arxiv.org/abs/2608.27128v1