Relative Variational Intrinsic Control

12/14/2020
by   Kate Baumli, et al.
0

In the absence of external rewards, agents can still learn useful behaviors by identifying and mastering a set of diverse skills within their environment. Existing skill learning methods use mutual information objectives to incentivize each skill to be diverse and distinguishable from the rest. However, if care is not taken to constrain the ways in which the skills are diverse, trivially diverse skill sets can arise. To ensure useful skill diversity, we propose a novel skill learning objective, Relative Variational Intrinsic Control (RVIC), which incentivizes learning skills that are distinguishable in how they change the agent's relationship to its environment. The resulting set of skills tiles the space of affordances available to the agent. We qualitatively analyze skill behaviors on multiple environments and show how RVIC skills are more useful than skills discovered by existing methods when used in hierarchical reinforcement learning.

READ FULL TEXT

page 2

page 4

page 5

page 6

research
05/21/2023

Unsupervised Discovery of Continuous Skills on a Sphere

Recently, methods for learning diverse skills to generate various behavi...
research
10/28/2021

Wasserstein Distance Maximizing Intrinsic Control

This paper deals with the problem of learning a skill-conditioned policy...
research
07/15/2019

Mutual Reinforcement Learning

Recently, collaborative robots have begun to train humans to achieve com...
research
06/06/2021

DisTop: Discovering a Topological representation to learn diverse and rewarding skills

The optimal way for a deep reinforcement learning (DRL) agent to explore...
research
07/21/2023

Diverse Offline Imitation via Fenchel Duality

There has been significant recent progress in the area of unsupervised s...
research
06/16/2023

Creating Multi-Level Skill Hierarchies in Reinforcement Learning

What is a useful skill hierarchy for an autonomous agent? We propose an ...
research
06/23/2020

ELSIM: End-to-end learning of reusable skills through intrinsic motivation

Taking inspiration from developmental learning, we present a novel reinf...

Please sign up or login with your details

Forgot password? Click here to reset