infrastructure
fact
neutral
Excessive weight decay drives scale-invariant weight norms past a critical boundary and destabilizes training, which explains why weight penalties cannot be made arbitrarily strong despite improving generalization
Neural and Evolutionary Computing28 Jul 2026