DeepAI AI Chat
Log In Sign Up

Solving a class of non-convex min-max games using adaptive momentum methods

by   Babak Barazandeh, et al.

Adaptive momentum methods have recently attracted a lot of attention for training of deep neural networks. They use an exponential moving average of past gradients of the objective function to update both search directions and learning rates. However, these methods are not suited for solving min-max optimization problems that arise in training generative adversarial networks. In this paper, we propose an adaptive momentum min-max algorithm that generalizes adaptive momentum methods to the non-convex min-max regime. Further, we establish non-asymptotic rates of convergence for the proposed algorithm when used in a reasonably broad class of non-convex min-max optimization problems. Experimental results illustrate its superior performance vis-a-vis benchmark methods for solving such problems.


A Decentralized Adaptive Momentum Method for Solving a Class of Min-Max Optimization Problems

Min-max saddle point games have recently been intensely studied, due to ...

Adaptive First-and Zeroth-order Methods for Weakly Convex Stochastic Optimization Problems

In this paper, we design and analyze a new family of adaptive subgradien...

A Hölderian backtracking method for min-max and min-min problems

We present a new algorithm to solve min-max or min-min problems out of t...

Towards Better Understanding of Adaptive Gradient Algorithms in Generative Adversarial Nets

Adaptive gradient algorithms perform gradient-based updates using the hi...

Training generative networks using random discriminators

In recent years, Generative Adversarial Networks (GANs) have drawn a lot...

Designing GANs: A Likelihood Ratio Approach

We are interested in the design of generative adversarial networks. The ...

Interior Point Methods with Adversarial Networks

We present a new methodology, called IPMAN, that combines interior point...