DeepAI AI Chat
Log In Sign Up

On Landscape of Lagrangian Functions and Stochastic Search for Constrained Nonconvex Optimization

06/13/2018
by   Zhehui Chen, et al.
Georgia Institute of Technology
0

We study constrained nonconvex optimization problems in machine learning, signal processing, and stochastic control. It is well-known that these problems can be rewritten to a minimax problem in a Lagrangian form. However, due to the lack of convexity, their landscape is not well understood and how to find the stable equilibria of the Lagrangian function is still unknown. To bridge the gap, we study the landscape of the Lagrangian function. Further, we define a special class of Lagrangian functions. They enjoy two properties: 1.Equilibria are either stable or unstable (Formal definition in Section 2); 2.Stable equilibria correspond to the global optima of the original problem. We show that a generalized eigenvalue (GEV) problem, including canonical correlation analysis and other problems, belongs to the class. Specifically, we characterize its stable and unstable equilibria by leveraging an invariant group and symmetric property (more details in Section 3). Motivated by these neat geometric structures, we propose a simple, efficient, and stochastic primal-dual algorithm solving the online GEV problem. Theoretically, we provide sufficient conditions, based on which we establish an asymptotic convergence rate and obtain the first sample complexity result for the online GEV problem by diffusion approximations, which are widely used in applied probability and stochastic control. Numerical results are provided to support our theory.

READ FULL TEXT
05/25/2016

NESTT: A Nonconvex Primal-Dual Splitting Method for Distributed and Stochastic Optimization

We study a stochastic and distributed algorithm for nonconvex problems w...
03/27/2020

Distributed and time-varying primal-dual dynamics via contraction analysis

In this paper, we provide a holistic analysis of the primal-dual dynamic...
01/05/2023

A first-order augmented Lagrangian method for constrained minimax optimization

In this paper we study a class of constrained minimax problems. In parti...
01/19/2022

Multiblock ADMM for nonsmooth nonconvex optimization with nonlinear coupling constraints

This paper considers a multiblock nonsmooth nonconvex optimization probl...
10/27/2020

Faster Lagrangian-Based Methods in Convex Optimization

In this paper, we aim at unifying, simplifying, and improving the conver...
01/23/2019

A Fully Stochastic Primal-Dual Algorithm

A new stochastic primal-dual algorithm for solving a composite optimizat...
08/16/2022

Large-Scale Minimization of the Pseudospectral Abscissa

This work concerns the minimization of the pseudospectral abscissa of a ...