Full Gradient DQN Reinforcement Learning: A Provably Convergent Scheme

03/10/2021
by   K. Avrachenkov, et al.
0

We analyze the DQN reinforcement learning algorithm as a stochastic approximation scheme using the o.d.e. (for `ordinary differential equation') approach and point out certain theoretical issues. We then propose a modified scheme called Full Gradient DQN (FG-DQN, for short) that has a sound theoretical basis and compare it with the original scheme on sample problems. We observe a better performance for FG-DQN.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
01/06/2021

Smoothed functional-based gradient algorithms for off-policy reinforcement learning

We consider the problem of control in an off-policy reinforcement learni...
research
11/13/2019

Asymptotics of Reinforcement Learning with Neural Networks

We prove that a single-layer neural network trained with the Q-learning ...
research
09/12/2016

A Threshold-based Scheme for Reinforcement Learning in Neural Networks

A generic and scalable Reinforcement Learning scheme for Artificial Neur...
research
03/04/2015

A Differential Equation for Modeling Nesterov's Accelerated Gradient Method: Theory and Insights

We derive a second-order ordinary differential equation (ODE) which is t...
research
07/20/2020

A Short Note on Soft-max and Policy Gradients in Bandits Problems

This is a short communication on a Lyapunov function argument for softma...
research
03/15/2023

Smoothed Q-learning

In Reinforcement Learning the Q-learning algorithm provably converges to...
research
01/18/2019

Derandomized Load Balancing using Random Walks on Expander Graphs

In a computing center with a huge amount of machines, when a job arrives...

Please sign up or login with your details

Forgot password? Click here to reset