DeepAI AI Chat
Log In Sign Up

Asynchronous Stochastic Approximation with Differential Inclusions

12/10/2011
by   Steven Perkins, et al.
University of Bristol
0

The asymptotic pseudo-trajectory approach to stochastic approximation of Benaim, Hofbauer and Sorin is extended for asynchronous stochastic approximations with a set-valued mean field. The asynchronicity of the process is incorporated into the mean field to produce convergence results which remain similar to those of an equivalent synchronous process. In addition, this allows many of the restrictive assumptions previously associated with asynchronous stochastic approximation to be removed. The framework is extended for a coupled asynchronous stochastic approximation process with set-valued mean fields. Two-timescales arguments are used here in a similar manner to the original work in this area by Borkar. The applicability of this approach is demonstrated through learning in a Markov decision process.

READ FULL TEXT

page 1

page 2

page 3

page 4

07/20/2018

A refined mean field approximation of synchronous discrete-time population models

Mean field approximation is a popular method to study the behaviour of s...
02/01/2020

Finite-Time Analysis of Asynchronous Stochastic Approximation and Q-Learning

We consider a general asynchronous Stochastic Approximation (SA) scheme ...
08/28/2021

Stochastic Approximation with Discontinuous Dynamics, Differential Inclusions, and Applications

This work develops new results for stochastic approximation algorithms. ...
02/06/2015

Stochastic recursive inclusion in two timescales with an application to the Lagrangian dual problem

In this paper we present a framework to analyze the asymptotic behavior ...
11/02/2021

Mean Field and Refined Mean Field Approximations for Heterogeneous Systems: It Works!

Mean field approximation is a powerful technique to study the performanc...
09/27/2022

Stability Via Adversarial Training of Neural Network Stochastic Control of Mean-Field Type

In this paper, we present an approach to neural network mean-field-type ...
06/20/2022

MF-OMO: An Optimization Formulation of Mean-Field Games

Theory of mean-field games (MFGs) has recently experienced an exponentia...