DeepAI AI Chat
Log In Sign Up

Asynchronous Stochastic Approximation with Differential Inclusions

by   Steven Perkins, et al.
University of Bristol

The asymptotic pseudo-trajectory approach to stochastic approximation of Benaim, Hofbauer and Sorin is extended for asynchronous stochastic approximations with a set-valued mean field. The asynchronicity of the process is incorporated into the mean field to produce convergence results which remain similar to those of an equivalent synchronous process. In addition, this allows many of the restrictive assumptions previously associated with asynchronous stochastic approximation to be removed. The framework is extended for a coupled asynchronous stochastic approximation process with set-valued mean fields. Two-timescales arguments are used here in a similar manner to the original work in this area by Borkar. The applicability of this approach is demonstrated through learning in a Markov decision process.


page 1

page 2

page 3

page 4


A refined mean field approximation of synchronous discrete-time population models

Mean field approximation is a popular method to study the behaviour of s...

Finite-Time Analysis of Asynchronous Stochastic Approximation and Q-Learning

We consider a general asynchronous Stochastic Approximation (SA) scheme ...

Stochastic Approximation with Discontinuous Dynamics, Differential Inclusions, and Applications

This work develops new results for stochastic approximation algorithms. ...

Stochastic recursive inclusion in two timescales with an application to the Lagrangian dual problem

In this paper we present a framework to analyze the asymptotic behavior ...

Mean Field and Refined Mean Field Approximations for Heterogeneous Systems: It Works!

Mean field approximation is a powerful technique to study the performanc...

Stability Via Adversarial Training of Neural Network Stochastic Control of Mean-Field Type

In this paper, we present an approach to neural network mean-field-type ...

MF-OMO: An Optimization Formulation of Mean-Field Games

Theory of mean-field games (MFGs) has recently experienced an exponentia...