ProgPrompt: Generating Situated Robot Task Plans using Large Language Models

09/22/2022
by   Ishika Singh, et al.
1

Task planning can require defining myriad domain knowledge about the world in which a robot needs to act. To ameliorate that effort, large language models (LLMs) can be used to score potential next actions during task planning, and even generate action sequences directly, given an instruction in natural language with no additional domain information. However, such methods either require enumerating all possible next steps for scoring, or generate free-form text that may contain actions not possible on a given robot in its current context. We present a programmatic LLM prompt structure that enables plan generation functional across situated environments, robot capabilities, and tasks. Our key insight is to prompt the LLM with program-like specifications of the available actions and objects in an environment, as well as with example programs that can be executed. We make concrete recommendations about prompt structure and generation constraints through ablation experiments, demonstrate state of the art success rates in VirtualHome household tasks, and deploy our method on a physical robot arm for tabletop tasks. Website at progprompt.github.io

READ FULL TEXT

page 1

page 2

page 6

research
10/10/2022

Generating Executable Action Plans with Environmentally-Aware Language Models

Large Language Models (LLMs) trained using massive text datasets have re...
research
11/01/2020

Semantic Task Planning for Service Robots in Open World

In this paper, we present a planning system based on semantic reasoning ...
research
03/24/2023

Errors are Useful Prompts: Instruction Guided Task Programming with Verifier-Assisted Iterative Prompting

Generating low-level robot task plans from high-level natural language i...
research
04/11/2022

Correcting Robot Plans with Natural Language Feedback

When humans design cost or goal specifications for robots, they often pr...
research
02/24/2023

Robot Behavior-Tree-Based Task Generation with Large Language Models

Nowadays, the behavior tree is gaining popularity as a representation fo...
research
05/18/2023

Generalized Planning in PDDL Domains with Pretrained Large Language Models

Recent work has considered whether large language models (LLMs) can func...
research
08/30/2023

Interactively Robot Action Planning with Uncertainty Analysis and Active Questioning by Large Language Model

The application of the Large Language Model (LLM) to robot action planni...

Please sign up or login with your details

Forgot password? Click here to reset