-
Average-case Acceleration for Bilinear Games and Normal Matrices
Advances in generative modeling and adversarial learning have given rise...
read it
-
A Unified Analysis of First-Order Methods for Smooth Games via Integral Quadratic Constraints
The theory of integral quadratic constraints (IQCs) allows the certifica...
read it
-
On the Suboptimality of Negative Momentum for Minimax Optimization
Smooth game optimization has recently attracted great interest in machin...
read it
-
A Laplacian Approach to ℓ_1-Norm Minimization
We propose a novel differentiable reformulation of the linearly-constrai...
read it
-
A Robust Framework for Analyzing Gradient-Based Dynamics in Bilinear Games
In this work, we establish a frequency-domain framework for analyzing gr...
read it
-
A Tight and Unified Analysis of Extragradient for a Whole Spectrum of Differentiable Games
We consider differentiable games: multi-objective minimization problems,...
read it
-
Gradientless Descent: High-Dimensional Zeroth-Order Optimization
Zeroth-order optimization is the process of minimizing an objective f(x)...
read it
Accelerating Smooth Games by Manipulating Spectral Shapes
We use matrix iteration theory to characterize acceleration in smooth games. We define the spectral shape of a family of games as the set containing all eigenvalues of the Jacobians of standard gradient dynamics in the family. Shapes restricted to the real line represent well-understood classes of problems, like minimization. Shapes spanning the complex plane capture the added numerical challenges in solving smooth games. In this framework, we describe gradient-based methods, such as extragradient, as transformations on the spectral shape. Using this perspective, we propose an optimal algorithm for bilinear games. For smooth and strongly monotone operators, we identify a continuum between convex minimization, where acceleration is possible using Polyak's momentum, and the worst case where gradient descent is optimal. Finally, going beyond first-order methods, we propose an accelerated version of consensus optimization.
READ FULL TEXT
Comments
There are no comments yet.