Solving Weakly-Convex-Weakly-Concave Saddle-Point Problems as Weakly-Monotone Variational Inequality
share
In this paper, we consider first-order algorithms for solving a class of non-convex non-concave min-max saddle-point problems, whose objective function is weakly convex (resp. weakly concave) in terms of the variable of minimization (resp. maximization). It has many important applications in machine learning, statistics, and operations research. One such example that attracts tremendous attention recently in machine learning is training Generative Adversarial Networks. We propose an algorithmic framework motivated by the proximal point method, which solve a sequence of strongly monotone variational inequalities constructed by adding a strongly monotone mapping to the original mapping with a periodically updated proximal center. By approximately solving each strongly monotone variational inequality, we prove that the solution obtained by the algorithm converges to a stationary solution of the original min-max problem. Iteration complexities are established for using different algorithms to solve the subproblems, including subgradient method, extragradient method and stochastic subgradient method. To the best of our knowledge, this is the first work that establishes the non-asymptotic convergence to a stationary point of a non-convex non-concave min-max problem.
READ FULL TEXT