Human and machine difficulty do not align in research design assessment - designs hardest for AI systems are not those where expert annotators disagree most

the designs that prove hardest for the system are not those on which expert annotators disagree most, pointing to two independent sources of task difficulty.

http://arxiv.org/abs/2608.27049v1