Self-Imitation Learning via Generalized Lower Bound Q-learning

06/12/2020
by   Yunhao Tang, et al.
1

Self-imitation learning motivated by lower-bound Q-learning is a novel and effective approach for off-policy learning. In this work, we propose a n-step lower bound which generalizes the original return-based lower-bound Q-learning, and introduce a new family of self-imitation learning algorithms. To provide a formal motivation for the potential performance gains provided by self-imitation learning, we show that n-step lower bound Q-learning achieves a trade-off between fixed point bias and contraction rate, drawing close connections to the popular uncorrected n-step Q-learning. We finally show that n-step lower bound Q-learning is a more robust alternative to return-based self-imitation learning and uncorrected n-step, over a wide range of continuous control benchmark tasks.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/26/2020

Episodic Self-Imitation Learning with Hindsight

Episodic self-imitation learning, a novel self-imitation algorithm with ...
research
02/27/2020

Provably Efficient Third-Person Imitation from Offline Observation

Domain adaptation in imitation learning represents an essential step tow...
research
04/13/2018

Learning Contracting Vector Fields For Stable Imitation Learning

We propose a new non-parametric framework for learning incrementally sta...
research
11/03/2021

Smooth Imitation Learning via Smooth Costs and Smooth Policies

Imitation learning (IL) is a popular approach in the continuous control ...
research
12/02/2021

Quantile Filtered Imitation Learning

We introduce quantile filtered imitation learning (QFIL), a novel policy...
research
12/01/2022

Multi-Task Imitation Learning for Linear Dynamical Systems

We study representation learning for efficient imitation learning over l...
research
03/16/2022

An Independently Learnable Hierarchical Model for Bilateral Control-Based Imitation Learning Applications

Recently, motion generation by machine learning has been actively resear...

Please sign up or login with your details

Forgot password? Click here to reset