Understanding Notions of Stationarity in Non-Smooth Optimization

06/26/2020
by   Jiajin Li, et al.
0

Many contemporary applications in signal processing and machine learning give rise to structured non-convex non-smooth optimization problems that can often be tackled by simple iterative methods quite effectively. One of the keys to understanding such a phenomenon—and, in fact, one of the very difficult conundrums even for experts—lie in the study of "stationary points" of the problem in question. Unlike smooth optimization, for which the definition of a stationary point is rather standard, there is a myriad of definitions of stationarity in non-smooth optimization. In this article, we give an introduction to different stationarity concepts for several important classes of non-convex non-smooth functions and discuss the geometric interpretations and further clarify the relationship among these different concepts. We then demonstrate the relevance of these constructions in some representative applications and how they could affect the performance of iterative methods for tackling these applications.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
01/24/2019

Perturbed Proximal Descent to Escape Saddle Points for Non-convex and Non-smooth Objective Functions

We consider the problem of finding local minimizers in non-convex and no...
research
01/30/2023

Single-Loop Switching Subgradient Methods for Non-Smooth Weakly Convex Optimization with Non-Smooth Convex Constraints

In this paper, we consider a general non-convex constrained optimization...
research
04/25/2018

Convergence guarantees for a class of non-convex and non-smooth optimization problems

We consider the problem of finding critical points of functions that are...
research
01/23/2019

Model Function Based Conditional Gradient Method with Armijo-like Line Search

The Conditional Gradient Method is generalized to a class of non-smooth ...
research
07/03/2019

Distributed Learning in Non-Convex Environments – Part I: Agreement at a Linear Rate

Driven by the need to solve increasingly complex optimization problems i...
research
05/03/2022

Smooth over-parameterized solvers for non-smooth structured optimization

Non-smooth optimization is a core ingredient of many imaging or machine ...
research
08/03/2023

Versatile Time-Frequency Representations Realized by Convex Penalty on Magnitude Spectrogram

Sparse time-frequency (T-F) representations have been an important resea...

Please sign up or login with your details

Forgot password? Click here to reset