Learning Language-Conditioned Robot Behavior from Offline Data and Crowd-Sourced Annotation

09/02/2021
by   Suraj Nair, et al.
1

We study the problem of learning a range of vision-based manipulation tasks from a large offline dataset of robot interaction. In order to accomplish this, humans need easy and effective ways of specifying tasks to the robot. Goal images are one popular form of task specification, as they are already grounded in the robot's observation space. However, goal images also have a number of drawbacks: they are inconvenient for humans to provide, they can over-specify the desired behavior leading to a sparse reward signal, or under-specify task information in the case of non-goal reaching tasks. Natural language provides a convenient and flexible alternative for task specification, but comes with the challenge of grounding language in the robot's observation space. To scalably learn this grounding we propose to leverage offline robot datasets (including highly sub-optimal, autonomously collected data) with crowd-sourced natural language labels. With this data, we learn a simple classifier which predicts if a change in state completes a language instruction. This provides a language-conditioned reward function that can then be used for offline multi-task RL. In our experiments, we find that on language-conditioned manipulation tasks our approach outperforms both goal-image specifications and language conditioned imitation techniques by more than 25 perform visuomotor tasks from natural language, such as "open the right drawer" and "move the stapler", on a Franka Emika Panda robot.

READ FULL TEXT

page 3

page 16

page 17

page 18

page 19

page 21

page 22

page 23

research
12/26/2020

Translating Natural Language Instructions to Computer Programs for Robot Manipulation

It is highly desirable for robots that work alongside humans to be able ...
research
07/26/2017

A Tale of Two DRAGGNs: A Hybrid Approach for Interpreting Action-Oriented and Goal-Oriented Instructions

Robots operating alongside humans in diverse, stochastic environments mu...
research
03/02/2023

Learning Language-Conditioned Deformable Object Manipulation with Graph Dynamics

Vision-based deformable object manipulation is a challenging problem in ...
research
09/26/2019

A Framework for Data-Driven Robotics

We present a framework for data-driven robotics that makes use of a larg...
research
06/19/2023

LARG, Language-based Automatic Reward and Goal Generation

Goal-conditioned and Multi-Task Reinforcement Learning (GCRL and MTRL) a...
research
10/18/2022

From Play to Policy: Conditional Behavior Generation from Uncurated Robot Data

While large-scale sequence modeling from offline data has led to impress...
research
04/13/2022

What Matters in Language Conditioned Robotic Imitation Learning

A long-standing goal in robotics is to build robots that can perform a w...

Please sign up or login with your details

Forgot password? Click here to reset