-
Policy-Gradient Algorithms Have No Guarantees of Convergence in Continuous Action and State Multi-Agent Settings
We show by counterexample that policy-gradient algorithms have no guaran...
read it
-
On the Convergence of Competitive, Multi-Agent Gradient-Based Learning
As learning algorithms are increasingly deployed in markets and other co...
read it
-
Convergence of Multi-Agent Learning with a Finite Step Size in General-Sum Games
Learning in a multi-agent system is challenging because agents are simul...
read it
-
Newton-based Policy Optimization for Games
Many learning problems involve multiple agents optimizing different inte...
read it
-
Tuned Hybrid Non-Uniform Subdivision Surfaces with Optimal Convergence Rates
This paper presents an enhanced version of our previous work, hybrid non...
read it
-
Performance Limits of Stochastic Sub-Gradient Learning, Part II: Multi-Agent Case
The analysis in Part I revealed interesting properties for subgradient l...
read it
-
Convergence Analysis of Deterministic Kernel-Based Quadrature Rules in Misspecified Settings
This paper presents convergence analysis of kernel-based quadrature rule...
read it
Convergence Analysis of Gradient-Based Learning with Non-Uniform Learning Rates in Non-Cooperative Multi-Agent Settings
Considering a class of gradient-based multi-agent learning algorithms in non-cooperative settings, we provide local convergence guarantees to a neighborhood of a stable local Nash equilibrium. In particular, we consider continuous games where agents learn in (i) deterministic settings with oracle access to their gradient and (ii) stochastic settings with an unbiased estimator of their gradient. Utilizing the minimum and maximum singular values of the game Jacobian, we provide finite-time convergence guarantees in the deterministic case. On the other hand, in the stochastic case, we provide concentration bounds guaranteeing that with high probability agents will converge to a neighborhood of a stable local Nash equilibrium in finite time. Different than other works in this vein, we also study the effects of non-uniform learning rates on the learning dynamics and convergence rates. We find that much like preconditioning in optimization, non-uniform learning rates cause a distortion in the vector field which can, in turn, change the rate of convergence and the shape of the region of attraction. The analysis is supported by numerical examples that illustrate different aspects of the theory. We conclude with discussion of the results and open questions.
READ FULL TEXT
Comments
There are no comments yet.