Important attention heads are scattered across layers without consistent layer-specific patterns in tabular transformers

A closer look at individual head importance scores across six attention layers reveals that important heads are scattered across layers, with no consistent layer-specific trends.

http://arxiv.org/abs/2608.27241v1