Meta-Gradient Reinforcement Learning

05/24/2018
by   Zhongwen Xu, et al.
0

The goal of reinforcement learning algorithms is to estimate and/or optimise the value function. However, unlike supervised learning, no teacher or oracle is available to provide the true value function. Instead, the majority of reinforcement learning algorithms estimate and/or optimise a proxy for the value function. This proxy is typically based on a sampled and bootstrapped approximation to the true value function, known as a return. The particular choice of return is one of the chief components determining the nature of the algorithm: the rate at which future rewards are discounted; when and how values should be bootstrapped; or even the nature of the rewards themselves. It is well-known that these decisions are crucial to the overall success of RL algorithms. We discuss a gradient-based meta-learning algorithm that is able to adapt the nature of the return, online, whilst interacting and learning from the environment. When applied to 57 games on the Atari 2600 environment over 200 million frames, our algorithm achieved a new state-of-the-art performance.

READ FULL TEXT
research
10/27/2017

Distributional Reinforcement Learning with Quantile Regression

In reinforcement learning an agent interacts with the environment by tak...
research
05/28/2019

Beyond Exponentially Discounted Sum: Automatic Learning of Return Function

In reinforcement learning, Return, which is the weighted accumulated fut...
research
01/05/2022

A Generalized Bootstrap Target for Value-Learning, Efficiently Combining Value and Feature Predictions

Estimating value functions is a core component of reinforcement learning...
research
02/05/2019

Separating value functions across time-scales

In many finite horizon episodic reinforcement learning (RL) settings, it...
research
02/24/2016

Learning values across many orders of magnitude

Most learning algorithms are not invariant to the scale of the function ...
research
06/11/2020

Surveys without Questions: A Reinforcement Learning Approach

The 'old world' instrument, survey, remains a tool of choice for firms t...
research
11/17/2020

Leveraging the Variance of Return Sequences for Exploration Policy

This paper introduces a method for constructing an upper bound for explo...

Please sign up or login with your details

Forgot password? Click here to reset