Head importance scoring can improve efficiency and reduce redundancy in transformer architectures

The proposed importance score can improve efficiency and redundancy within transformer architectures.

http://arxiv.org/abs/2608.27241v1