Vision-language models contain Visual Retrieval Heads (VRHs), a small subset of about 1.7-2.6% of attention heads that are causally responsible for grounding text descriptions to image regions

Visual Retrieval Heads (VRHs), a small subset of attention heads (about 1.7-2.6%) that are causally responsible for grounding text descriptions to image regions

http://arxiv.org/abs/2608.27417v1