Batch prompting makes LLM inference more efficient by processing multiple instances simultaneously but suffers from unpredictable downstream task performance

batch prompting makes large language model inference more efficient by processing multiple instances simultaneously, it suffers from unpredictable downstream task performance

http://arxiv.org/abs/2608.27038v1