Language-guided Task Adaptation for Imitation Learning

01/24/2023
by   Prasoon Goyal, et al.
0

We introduce a novel setting, wherein an agent needs to learn a task from a demonstration of a related task with the difference between the tasks communicated in natural language. The proposed setting allows reusing demonstrations from other tasks, by providing low effort language descriptions, and can also be used to provide feedback to correct agent errors, which are both important desiderata for building intelligent agents that assist humans in daily tasks. To enable progress in this proposed setting, we create two benchmarks – Room Rearrangement and Room Navigation – that cover a diverse set of task adaptations. Further, we propose a framework that uses a transformer-based model to reason about the entities in the tasks and their relationships, to learn a policy for the target task

READ FULL TEXT

page 15

page 16

research
06/05/2021

Zero-shot Task Adaptation using Natural Language

Imitation learning and instruction-following are two common approaches t...
research
06/07/2019

Watch, Try, Learn: Meta-Learning from Demonstrations and Reward

Imitation learning allows agents to learn complex behaviors from demonst...
research
11/19/2018

Guiding Policies with Language via Meta-Learning

Behavioral skills or policies for autonomous agents are conventionally l...
research
12/10/2018

Vision-based Navigation with Language-based Assistance via Imitation Learning with Indirect Intervention

We present Vision-based Navigation with Language-based Assistance (VNLA)...
research
10/09/2019

Fast Task-Adaptation for Tasks Labeled Using Natural Language in Reinforcement Learning

Over its lifetime, a reinforcement learning agent is often tasked with d...
research
12/07/2021

Combining Learning from Human Feedback and Knowledge Engineering to Solve Hierarchical Tasks in Minecraft

Real-world tasks of interest are generally poorly defined by human-reada...
research
03/29/2023

EgoTV: Egocentric Task Verification from Natural Language Task Descriptions

To enable progress towards egocentric agents capable of understanding ev...

Please sign up or login with your details

Forgot password? Click here to reset