Incrementally Learning Functions of the Return

07/05/2019 ∙ by Brendan Bennett, et al. ∙ 0 ∙ share

Temporal difference methods enable efficient estimation of value functions in reinforcement learning in an incremental fashion, and are of broader interest because they correspond learning as observed in biological systems. Standard value functions correspond to the expected value of a sum of discounted returns. While this formulation is often sufficient for many purposes, it would often be useful to be able to represent functions of the return as well. Unfortunately, most such functions cannot be estimated directly using TD methods. We propose a means of estimating functions of the return using its moments, which can be learned online using a modified TD algorithm. The moments of the return are then used as part of a Taylor expansion to approximate analytic functions of the return.

READ FULL TEXT

Directly Estimating the Variance of the λ-Return Using Temporal-Difference Methods

This paper investigates estimating the variance of a temporal-difference...
01/25/2018 ∙ by Craig Sherstan, et al. ∙ 0 ∙ share

read it
Beyond Exponentially Discounted Sum: Automatic Learning of Return Function

In reinforcement learning, Return, which is the weighted accumulated fut...
05/28/2019 ∙ by Yufei Wang, et al. ∙ 0 ∙ share

read it
Parametric Return Density Estimation for Reinforcement Learning

Most conventional Reinforcement Learning (RL) algorithms aim to optimize...
03/15/2012 ∙ by Tetsuro Morimura, et al. ∙ 0 ∙ share

read it
Off-policy Multi-step Q-learning

In the past few years, off-policy reinforcement learning methods have sh...
09/30/2019 ∙ by Gabriel Kalweit, et al. ∙ 0 ∙ share

read it
Return to Bali

This paper gives an overview of the project Return to Bali that seeks to...
09/01/2020 ∙ by Marc Böhlen, et al. ∙ 0 ∙ share

read it
Simulating an infinite mean waiting time

We consider a hybrid method to simulate the return time to the initial s...
07/23/2019 ∙ by Krzysztof Bartoszek, et al. ∙ 0 ∙ share

read it
Mixture of Step Returns in Bootstrapped DQN

The concept of utilizing multi-step returns for updating value functions...
07/16/2020 ∙ by Po-Han Chiang, et al. ∙ 0 ∙ share

read it

Incrementally Learning Functions of the Return

Authors

Directly Estimating the Variance of the λ-Return Using Temporal-Difference Methods

Beyond Exponentially Discounted Sum: Automatic Learning of Return Function

Parametric Return Density Estimation for Reinforcement Learning

Off-policy Multi-step Q-learning

Return to Bali

Simulating an infinite mean waiting time

Mixture of Step Returns in Bootstrapped DQN

This week in AI