Prospection: Interpretable Plans From Language By Predicting the Future

03/20/2019
by   Chris Paxton, et al.
8

High-level human instructions often correspond to behaviors with multiple implicit steps. In order for robots to be useful in the real world, they must be able to to reason over both motions and intermediate goals implied by human instructions. In this work, we propose a framework for learning representations that convert from a natural-language command to a sequence of intermediate goals for execution on a robot. A key feature of this framework is prospection, training an agent not just to correctly execute the prescribed command, but to predict a horizon of consequences of an action before taking it. We demonstrate the fidelity of plans generated by our framework when interpreting real, crowd-sourced natural language commands for a robot in simulated scenes.

READ FULL TEXT

page 3

page 4

page 5

page 6

page 7

page 9

page 10

page 11

research
11/20/2016

Generating machine-executable plans from end-user's natural-language instructions

It is critical for advanced manufacturing machines to autonomously execu...
research
09/08/2023

Incremental Learning of Humanoid Robot Behavior from Natural Interaction and Large Language Models

Natural-language dialog is key for intuitive human-robot interaction. It...
research
11/01/2022

Learning to Solve Voxel Building Embodied Tasks from Pixels and Natural Language Instructions

The adoption of pre-trained language models to generate action plans for...
research
06/09/2016

Understanding User Instructions by Utilizing Open Knowledge for Service Robots

Understanding user instructions in natural language is an active researc...
research
11/08/2017

Learning to Imagine Manipulation Goals for Robot Task Planning

Prospection is an important part of how humans come up with new task pla...
research
10/12/2022

Interactive Language: Talking to Robots in Real Time

We present a framework for building interactive, real-time, natural lang...
research
04/11/2022

Correcting Robot Plans with Natural Language Feedback

When humans design cost or goal specifications for robots, they often pr...

Please sign up or login with your details

Forgot password? Click here to reset