Attention head importance varies considerably across different tabular datasets, unlike in image and language domains

In contrast to the image and language domains, the importance of individual attention heads varies considerably across tabular datasets with different schemas and feature spaces.

http://arxiv.org/abs/2608.27241v1