Eigenoption Discovery through the Deep Successor Representation

10/30/2017
by   Marlos C. Machado, et al.
0

Options in reinforcement learning allow agents to hierarchically decompose a task into subtasks, having the potential to speed up learning and planning. However, autonomously learning effective sets of options is still a major challenge in the field. In this paper we focus on the recently introduced idea of using representation learning methods to guide the option discovery process. Specifically, we look at eigenoptions, options obtained from representations that encode diffusive information flow in the environment. We extend the existing algorithms for eigenoption discovery to settings with stochastic transitions and in which handcrafted features are not available. We propose an algorithm that discovers eigenoptions while learning non-linear state representations from raw pixels. It exploits recent successes in the deep reinforcement learning literature and the equivalence between proto-value functions and the successor representation. We use traditional tabular domains to provide intuition about our approach and Atari 2600 games to demonstrate its potential.

READ FULL TEXT

page 3

page 8

page 14

page 15

page 16

page 17

page 18

page 19

research
02/07/2022

Reward-Respecting Subtasks for Model-Based Reinforcement Learning

To achieve the ambitious goals of artificial intelligence, reinforcement...
research
05/14/2019

Successor Options: An Option Discovery Framework for Reinforcement Learning

The options framework in reinforcement learning models the notion of a s...
research
01/06/2020

Optimal Options for Multi-Task Reinforcement Learning Under Time Constraints

Reinforcement learning can greatly benefit from the use of options as a ...
research
12/11/2017

The Eigenoption-Critic Framework

Eigenoptions (EOs) have been recently introduced as a promising idea for...
research
02/28/2012

Relational Reinforcement Learning in Infinite Mario

Relational representations in reinforcement learning allow for the use o...
research
12/12/2012

The Thing That We Tried Didn't Work Very Well : Deictic Representation in Reinforcement Learning

Most reinforcement learning methods operate on propositional representat...
research
03/12/2020

Option Discovery in the Absence of Rewards with Manifold Analysis

Options have been shown to be an effective tool in reinforcement learnin...

Please sign up or login with your details

Forgot password? Click here to reset