Successor Features for Transfer in Reinforcement Learning

06/16/2016
by   Andre Barreto, et al.
0

Transfer in reinforcement learning refers to the notion that generalization should occur not only within a task but also across tasks. Our focus is on transfer where the reward functions vary across tasks while the environment's dynamics remain the same. The method we propose rests on two key ideas: "successor features," a value function representation that decouples the dynamics of the environment from the rewards, and "generalized policy improvement," a generalization of dynamic programming's policy improvement step that considers a set of policies rather than a single one. Put together, the two ideas lead to an approach that integrates seamlessly within the reinforcement learning framework and allows transfer to take place between tasks without any restriction. The proposed method also provides performance guarantees for the transferred policy even before any learning has taken place. We derive two theorems that set our approach in firm theoretical ground and present experiments that show that it successfully promotes transfer in practice.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
01/30/2019

Transfer in Deep Reinforcement Learning Using Successor Features and Generalised Policy Improvement

The ability to transfer skills across tasks has the potential to scale u...
research
04/11/2018

Universal Successor Representations for Transfer Reinforcement Learning

The objective of transfer reinforcement learning is to generalize from a...
research
11/04/2021

Successor Feature Neural Episodic Control

A longstanding goal in reinforcement learning is to build intelligent ag...
research
07/04/2018

Transfer with Model Features in Reinforcement Learning

A key question in Reinforcement Learning is which representation an agen...
research
07/18/2021

A New Representation of Successor Features for Transfer across Dissimilar Environments

Transfer in reinforcement learning is usually achieved through generalis...
research
12/30/2021

Constructing a Good Behavior Basis for Transfer using Generalized Policy Updates

We study the problem of learning a good set of policies, so that when co...
research
09/26/2019

High-Dimensional Control Using Generalized Auxiliary Tasks

A long-standing challenge in reinforcement learning is the design of fun...

Please sign up or login with your details

Forgot password? Click here to reset