Bellman Error Based Feature Generation using Random Projections on Sparse Spaces

07/23/2012
by   Mahdi Milani Fard, et al.
0

We address the problem of automatic generation of features for value function approximation. Bellman Error Basis Functions (BEBFs) have been shown to improve the error of policy evaluation with function approximation, with a convergence rate similar to that of value iteration. We propose a simple, fast and robust algorithm based on random projections to generate BEBFs for sparse feature spaces. We provide a finite sample analysis of the proposed method, and prove that projections logarithmic in the dimension of the original space are enough to guarantee contraction in the error. Empirical results demonstrate the strength of this method.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/25/2018

Finite Sample Analysis of LSTD with Random Projections and Eligibility Traces

Policy evaluation with linear function approximation is an important pro...
research
02/03/2020

Finite-Sample Analysis of Stochastic Approximation Using Smooth Convex Envelopes

Stochastic Approximation (SA) is a popular approach for solving fixed po...
research
08/22/2019

On Convergence Rate of Adaptive Multiscale Value Function Approximation For Reinforcement Learning

In this paper, we propose a generic framework for devising an adaptive a...
research
08/28/2018

High-confidence error estimates for learned value functions

Estimating the value function for a fixed policy is a fundamental proble...
research
06/11/2013

Stochastic approximation for speeding up LSTD (and LSPI)

We propose a stochastic approximation (SA) based method with randomizati...
research
04/05/2021

Alternating projections with applications to Gerchberg-Saxton error reduction

We consider convergence of alternating projections between non-convex se...
research
03/04/2021

Function Approximation via Sparse Random Features

Random feature methods have been successful in various machine learning ...

Please sign up or login with your details

Forgot password? Click here to reset