Lifted Primal-Dual Method for Bilinearly Coupled Smooth Minimax Optimization

We study the bilinearly coupled minimax problem: min_xmax_y f(x) + y^⊤ A x - h(y), where f and h are both strongly convex smooth functions and admit first-order gradient oracles. Surprisingly, no known first-order algorithms have hitherto achieved the lower complexity bound of Ω((√(L_x/μ_x) + A/√(μ_x μ_y) + √(L_y/μ_y)) log(1/ε)) for solving this problem up to an ε primal-dual gap in the general parameter regime, where L_x, L_y,μ_x,μ_y are the corresponding smoothness and strongly convexity constants. We close this gap by devising the first optimal algorithm, the Lifted Primal-Dual (LPD) method. Our method lifts the objective into an extended form that allows both the smooth terms and the bilinear term to be handled optimally and seamlessly with the same primal-dual framework. Besides optimality, our method yields a desirably simple single-loop algorithm that uses only one gradient oracle call per iteration. Moreover, when f is just convex, the same algorithm applied to a smoothed objective achieves the nearly optimal iteration complexity. We also provide a direct single-loop algorithm, using the LPD method, that achieves the iteration complexity of O(√(L_x/ε) + A/√(μ_y ε) + √(L_y/ε)). Numerical experiments on quadratic minimax problems and policy evaluation problems further demonstrate the fast convergence of our algorithm in practice.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
12/09/2022

Primal Dual Alternating Proximal Gradient Algorithms for Nonsmooth Nonconvex Minimax Problems with Coupled Linear Constraints

Nonconvex minimax problems have attracted wide attention in machine lear...
research
09/05/2018

Solving Non-smooth Constrained Programs with Lower Complexity than O(1/ε): A Primal-Dual Homotopy Smoothing Approach

We propose a new primal-dual homotopy smoothing algorithm for a linearly...
research
06/01/2018

Efficient, Certifiably Optimal High-Dimensional Clustering

We consider SDP relaxation methods for data and variable clustering prob...
research
01/24/2019

SAGA with Arbitrary Sampling

We study the problem of minimizing the average of a very large number of...
research
07/13/2016

Homotopy Smoothing for Non-Smooth Problems with Lower Complexity than O(1/ε)

In this paper, we develop a novel homoto py smoothing (HOPS) algorithm...
research
07/07/2014

The Primal-Dual Hybrid Gradient Method for Semiconvex Splittings

This paper deals with the analysis of a recent reformulation of the prim...
research
07/29/2023

Faster Stochastic Algorithms for Minimax Optimization under Polyak–Łojasiewicz Conditions

This paper considers stochastic first-order algorithms for minimax optim...

Please sign up or login with your details

Forgot password? Click here to reset