-
Accelerating Smooth Games by Manipulating Spectral Shapes
We use matrix iteration theory to characterize acceleration in smooth ga...
read it
-
Stochastic Hamiltonian Gradient Methods for Smooth Games
The success of adversarial formulations in machine learning has brought ...
read it
-
A Unified Analysis of First-Order Methods for Smooth Games via Integral Quadratic Constraints
The theory of integral quadratic constraints (IQCs) allows the certifica...
read it
-
An optimal gradient method for smooth (possibly strongly) convex minimization
We present an optimal gradient method for smooth (possibly strongly) con...
read it
-
Convergence Behaviour of Some Gradient-Based Methods on Bilinear Zero-Sum Games
Min-max formulations have attracted great attention in the ML community ...
read it
-
Asymptotic Theory of Eigenvectors for Large Random Matrices
Characterizing the exact asymptotic distributions of high-dimensional ei...
read it
-
Faster Tensor Canonicalization
The Butler-Portugal algorithm for obtaining the canonical form of a tens...
read it
Average-case Acceleration for Bilinear Games and Normal Matrices
Advances in generative modeling and adversarial learning have given rise to renewed interest in smooth games. However, the absence of symmetry in the matrix of second derivatives poses challenges that are not present in the classical minimization framework. While a rich theory of average-case analysis has been developed for minimization problems, little is known in the context of smooth games. In this work we take a first step towards closing this gap by developing average-case optimal first-order methods for a subset of smooth games. We make the following three main contributions. First, we show that for zero-sum bilinear games the average-case optimal method is the optimal method for the minimization of the Hamiltonian. Second, we provide an explicit expression for the optimal method corresponding to normal matrices, potentially non-symmetric. Finally, we specialize it to matrices with eigenvalues located in a disk and show a provable speed-up compared to worst-case optimal algorithms. We illustrate our findings through benchmarks with a varying degree of mismatch with our assumptions.
READ FULL TEXT
Comments
There are no comments yet.