DeepAI
Log In Sign Up

Universal Value Density Estimation for Imitation Learning and Goal-Conditioned Reinforcement Learning

02/15/2020
by   Yannick Schroecker, et al.
0

This work considers two distinct settings: imitation learning and goal-conditioned reinforcement learning. In either case, effective solutions require the agent to reliably reach a specified state (a goal), or set of states (a demonstration). Drawing a connection between probabilistic long-term dynamics and the desired value function, this work introduces an approach which utilizes recent advances in density estimation to effectively learn to reach a given state. As our first contribution, we use this approach for goal-conditioned reinforcement learning and show that it is both efficient and does not suffer from hindsight bias in stochastic domains. As our second contribution, we extend the approach to imitation learning and show that it achieves state-of-the art demonstration sample-efficiency on standard benchmark tasks.

READ FULL TEXT

page 1

page 2

page 3

page 4

12/12/2019

Learning To Reach Goals Without Reinforcement Learning

Imitation learning algorithms provide a simple and straightforward appro...
06/13/2019

Goal-conditioned Imitation Learning

Designing rewards for Reinforcement Learning (RL) is challenging because...
02/26/2018

Reinforcement and Imitation Learning for Diverse Visuomotor Skills

We propose a model-free deep reinforcement learning method that leverage...
09/26/2022

Understanding Hindsight Goal Relabeling Requires Rethinking Divergence Minimization

Hindsight goal relabeling has become a foundational technique for multi-...
05/16/2022

Generalizing to New Tasks via One-Shot Compositional Subgoals

The ability to generalize to previously unseen tasks with little to no s...
07/09/2019

Hybrid system identification using switching density networks

Behaviour cloning is a commonly used strategy for imitation learning and...
08/17/2022

Metric Residual Networks for Sample Efficient Goal-Conditioned Reinforcement Learning

Goal-conditioned reinforcement learning (GCRL) has a wide range of poten...