DeepAI AI Chat
Log In Sign Up

Exploring with Sticky Mittens: Reinforcement Learning with Expert Interventions via Option Templates

02/25/2022
by   Souradeep Dutta, et al.
Children's Hospital of Philadelphia
University of Pennsylvania
9

Environments with sparse rewards and long horizons pose a significant challenge for current reinforcement learning algorithms. A key feature enabling humans to learn challenging control tasks is that they often receive expert intervention that enables them to understand the high-level structure of the task before mastering low-level control actions. We propose a framework for leveraging expert intervention to solve long-horizon reinforcement learning tasks. We consider option templates, which are specifications encoding a potential option that can be trained using reinforcement learning. We formulate expert intervention as allowing the agent to execute option templates before learning an implementation. This enables them to use an option, before committing costly resources to learning it. We evaluate our approach on three challenging reinforcement learning problems, showing that it outperforms state of-the-art approaches by an order of magnitude. Project website at https://sites.google.com/view/stickymittens

READ FULL TEXT
10/11/2022

DHRL: A Graph-Based Approach for Long-Horizon and Sparse Hierarchical Reinforcement Learning

Hierarchical Reinforcement Learning (HRL) has made notable progress in c...
03/29/2022

Demonstration-Bootstrapped Autonomous Practicing via Multi-Task Reinforcement Learning

Reinforcement learning systems have the potential to enable continuous i...
10/19/2022

When to Ask for Help: Proactive Interventions in Autonomous Reinforcement Learning

A long-term goal of reinforcement learning is to design agents that can ...
06/10/2021

Adversarial Option-Aware Hierarchical Imitation Learning

It has been a challenge to learning skills for an agent from long-horizo...
12/06/2021

Hierarchical Reinforcement Learning with Timed Subgoals

Hierarchical reinforcement learning (HRL) holds great potential for samp...
01/30/2023

Hierarchical Imitation Learning with Vector Quantized Models

The ability to plan actions on multiple levels of abstraction enables in...
10/16/2022

Towards an Interpretable Hierarchical Agent Framework using Semantic Goals

Learning to solve long horizon temporally extended tasks with reinforcem...