Standard DPO loss values are not comparable across different β values, with runs having nearly identical loss curves potentially differing several-fold in KL divergence from the reference model

standard DPO loss values are not comparable across $β$: runs with nearly identical loss curves can differ several-fold in KL divergence from the reference model

http://arxiv.org/abs/2608.27032v1