Show me the Way: Intrinsic Motivation from Demonstrations

06/23/2020
by   Léonard Hussenot, et al.
0

The study of exploration in Reinforcement Learning (RL) has a long history but it remains an unsolved problem. Recent approaches applied to Deep RL are based on the concept of intrinsic motivation and are implemented in the shape of an exploration bonus, added to the environment reward, that encourages visiting exhaustively the whole state-action space as fast as possible. This approach is supported by the vast theory of RL for which convergence to optimality assumes exhaustive exploration. Yet, Human Beings and mammals do not exhaustively explore the world and their motivation is not only based on novelty but also on diverse other factors (e.g., curiosity, fun, style, pleasure, safety, competition, etc.). They optimize for life-long learning and train to learn transferable skills in playgrounds without obvious goals. They also apply innate or learned priors to save time and stay safe. For these reasons, we propose a method for learning an exploration bonus from demonstrations that could transfer these motivations to an artificial agent without explicitly modeling them. Using an inverse RL approach, we show that different exploration behaviors can be learnt and efficiently used by RL agents to solve tasks for which exhaustive exploration is prohibitive.

READ FULL TEXT
research
07/05/2023

First-Explore, then Exploit: Meta-Learning Intelligent Exploration

Standard reinforcement learning (RL) agents never intelligently explore ...
research
09/19/2022

An information-theoretic perspective on intrinsic motivation in reinforcement learning: a survey

The reinforcement learning (RL) research area is very active, with an im...
research
12/06/2022

First Go, then Post-Explore: the Benefits of Post-Exploration in Intrinsic Motivation

Go-Explore achieved breakthrough performance on challenging reinforcemen...
research
08/26/2020

Intrinsic Motivation in Object-Action-Outcome Blending Latent Space

One effective approach for equipping artificial agents with sensorimotor...
research
03/29/2022

When to Go, and When to Explore: The Benefit of Post-Exploration in Intrinsic Motivation

Go-Explore achieved breakthrough performance on challenging reinforcemen...
research
03/31/2023

Accelerating exploration and representation learning with offline pre-training

Sequential decision-making agents struggle with long horizon tasks, sinc...
research
07/01/2018

Beyond Winning and Losing: Modeling Human Motivations and Behaviors Using Inverse Reinforcement Learning

In recent years, reinforcement learning (RL) methods have been applied t...

Please sign up or login with your details

Forgot password? Click here to reset