Prompting Multi-modal LLMs to explicitly estimate system state and predict state transitions ensures task-convergent planning in human-robot collaboration

By forecasting future states alongside actions, STEP ensures task-convergent planning while also providing additional assistance parameters necessary for executing the predicted actions

http://arxiv.org/abs/2608.27225v1