Planning to Explore via Self-Supervised World Models

05/12/2020
by   Ramanan Sekar, et al.
2

Reinforcement learning allows solving complex tasks, however, the learning tends to be task-specific and the sample efficiency remains a challenge. We present Plan2Explore, a self-supervised reinforcement learning agent that tackles both these challenges through a new approach to self-supervised exploration and fast adaptation to new tasks, which need not be known during exploration. During exploration, unlike prior methods which retrospectively compute the novelty of observations after the agent has already reached them, our agent acts efficiently by leveraging planning to seek out expected future novelty. After exploration, the agent quickly adapts to multiple downstream tasks in a zero or a few-shot manner. We evaluate on challenging control tasks from high-dimensional image inputs. Without any training supervision or task-specific interaction, Plan2Explore outperforms prior self-supervised exploration methods, and in fact, almost matches the performances oracle which has access to rewards. Videos and code at https://ramanans1.github.io/plan2explore/

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/22/2022

Curious Exploration via Structured World Models Yields Zero-Shot Object Manipulation

It has been a long-standing dream to design artificial agents that explo...
research
08/26/2021

Glimpse-Attend-and-Explore: Self-Attention for Active Visual Exploration

Active visual exploration aims to assist an agent with a limited field o...
research
06/21/2023

Optimistic Active Exploration of Dynamical Systems

Reinforcement learning algorithms commonly seek to optimize policies for...
research
08/03/2022

Character Generation through Self-Supervised Vectorization

The prevalent approach in self-supervised image generation is to operate...
research
10/23/2022

Learning General World Models in a Handful of Reward-Free Deployments

Building generally capable agents is a grand challenge for deep reinforc...
research
06/27/2019

Supervise Thyself: Examining Self-Supervised Representations in Interactive Environments

Self-supervised methods, wherein an agent learns representations solely ...
research
01/03/2019

Self-supervised Learning of Image Embedding for Continuous Control

Operating directly from raw high dimensional sensory inputs like images ...

Please sign up or login with your details

Forgot password? Click here to reset