Learning High-level Representations from Demonstrations

02/19/2018
by   Garrett Andersen, et al.
0

Hierarchical learning (HL) is key to solving complex sequential decision problems with long horizons and sparse rewards. It allows learning agents to break-up large problems into smaller, more manageable subtasks. A common approach to HL, is to provide the agent with a number of high-level skills that solve small parts of the overall problem. A major open question, however, is how to identify a suitable set of reusable skills. We propose a principled approach that uses human demonstrations to infer a set of subgoals based on changes in the demonstration dynamics. Using these subgoals, we decompose the learning problem into an abstract high-level representation and a set of low-level subtasks. The abstract description captures the overall problem structure, while subtasks capture desired skills. We demonstrate that we can jointly optimize over both levels of learning. We show that the resulting method significantly outperforms previous baselines on two challenging problems: the Atari 2600 game Montezuma's Revenge, and a simulated robotics problem moving the ant robot through a maze.

READ FULL TEXT

page 5

page 6

research
10/10/2019

Hierarchical Reinforcement Learning with Advantage-Based Auxiliary Rewards

Hierarchical Reinforcement Learning (HRL) is a promising approach to sol...
research
09/26/2018

Scaling simulation-to-real transfer by learning composable robot skills

We present a novel solution to the problem of simulation-to-real transfe...
research
06/13/2023

Skill Disentanglement for Imitation Learning from Suboptimal Demonstrations

Imitation learning has achieved great success in many sequential decisio...
research
10/04/2021

Skill Induction and Planning with Latent Language

We present a framework for learning hierarchical policies from demonstra...
research
09/21/2022

Hierarchical Decision Transformer

Sequence models in reinforcement learning require task knowledge to esti...
research
07/12/2018

Automatically Composing Representation Transformations as a Means for Generalization

How can we build a learner that can capture the essence of what makes a ...
research
03/22/2019

Deep Hierarchical Reinforcement Learning Based Recommendations via Multi-goals Abstraction

The recommender system is an important form of intelligent application, ...

Please sign up or login with your details

Forgot password? Click here to reset