A New Representation of Successor Features for Transfer across Dissimilar Environments

07/18/2021
by   Majid Abdolshah, et al.
6

Transfer in reinforcement learning is usually achieved through generalisation across tasks. Whilst many studies have investigated transferring knowledge when the reward function changes, they have assumed that the dynamics of the environments remain consistent. Many real-world RL problems require transfer among environments with different dynamics. To address this problem, we propose an approach based on successor features in which we model successor feature functions with Gaussian Processes permitting the source successor features to be treated as noisy measurements of the target successor feature function. Our theoretical analysis proves the convergence of this approach as well as the bounded error on modelling successor feature functions with Gaussian Processes in environments with both different dynamics and rewards. We demonstrate our method on benchmark datasets and show that it outperforms current baselines.

READ FULL TEXT
research
10/29/2021

Xi-Learning: Successor Feature Transfer Learning for General Reward Functions

Transfer in Reinforcement Learning aims to improve learning performance ...
research
07/31/2017

Advantages and Limitations of using Successor Features for Transfer in Reinforcement Learning

One question central to Reinforcement Learning is how to learn a feature...
research
06/16/2016

Successor Features for Transfer in Reinforcement Learning

Transfer in reinforcement learning refers to the notion that generalizat...
research
07/04/2018

Transfer with Model Features in Reinforcement Learning

A key question in Reinforcement Learning is which representation an agen...
research
04/27/2018

Decoupling Dynamics and Reward for Transfer Learning

Current reinforcement learning (RL) methods can successfully learn singl...
research
05/26/2020

Time-Variant Variational Transfer for Value Functions

In most transfer learning approaches to reinforcement learning (RL) the ...
research
10/17/2019

Single Episode Policy Transfer in Reinforcement Learning

Transfer and adaptation to new unknown environmental dynamics is a key c...

Please sign up or login with your details

Forgot password? Click here to reset