DeepAI AI Chat
Log In Sign Up

Bayesian Curiosity for Efficient Exploration in Reinforcement Learning

by   Tom Blau, et al.
The University of Sydney

Balancing exploration and exploitation is a fundamental part of reinforcement learning, yet most state-of-the-art algorithms use a naive exploration protocol like ϵ-greedy. This contributes to the problem of high sample complexity, as the algorithm wastes effort by repeatedly visiting parts of the state space that have already been explored. We introduce a novel method based on Bayesian linear regression and latent space embedding to generate an intrinsic reward signal that encourages the learning agent to seek out unexplored parts of the state space. This method is computationally efficient, simple to implement, and can extend any state-of-the-art reinforcement learning algorithm. We evaluate the method on a range of algorithms and challenging control tasks, on both simulated and physical robots, demonstrating how the proposed method can significantly improve sample complexity.


page 1

page 4


Information Maximizing Exploration with a Latent Dynamics Model

All reinforcement learning algorithms must handle the trade-off between ...

Learning-Driven Exploration for Reinforcement Learning

Deep reinforcement learning algorithms have been shown to learn complex ...

Mo' States Mo' Problems: Emergency Stop Mechanisms from Observation

In many environments, only a relatively small subset of the complete sta...

Kinematic State Abstraction and Provably Efficient Rich-Observation Reinforcement Learning

We present an algorithm, HOMER, for exploration and reinforcement learni...

Adaptive Sample-Efficient Blackbox Optimization via ES-active Subspaces

We present a new algorithm ASEBO for conducting optimization of high-dim...

Fractional Moments on Bandit Problems

Reinforcement learning addresses the dilemma between exploration to find...

Control-Tutored Reinforcement Learning

We introduce a control-tutored reinforcement learning (CTRL) algorithm. ...