Optimization of Inf-Convolution Regularized Nonconvex Composite Problems

03/27/2019
by   Emanuel Laude, et al.
0

In this work, we consider nonconvex composite problems that involve inf-convolution with a Legendre function, which gives rise to an anisotropic generalization of the proximal mapping and Moreau-envelope. In a convex setting such problems can be solved via alternating minimization of a splitting formulation, where the consensus constraint is penalized with a Legendre function. In contrast, for nonconvex models it is in general unclear that this approach yields stationary points to the infimal convolution problem. To this end we analytically investigate local regularity properties of the Moreau-envelope function under prox-regularity, which allows us to establish the equivalence between stationary points of the splitting model and the original inf-convolution model. We apply our theory to characterize stationary points of the penalty objective, which is minimized by the elastic averaging SGD (EASGD) method for distributed training. Numerically, we demonstrate the practical relevance of the proposed approach on the important task of distributed training of deep neural networks.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/03/2014

Global convergence of splitting methods for nonconvex composite optimization

We consider the problem of minimizing the sum of a smooth function h wit...
research
09/01/2011

Nonconvex proximal splitting: batch and incremental algorithms

Within the unmanageably large class of nonconvex optimization, we consid...
research
05/25/2022

Complexity-Optimal and Curvature-Free First-Order Methods for Finding Stationary Points of Composite Optimization Problems

This paper develops and analyzes an accelerated proximal descent method ...
research
02/20/2018

Composite Optimization by Nonconvex Majorization-Minimization

Many tasks in imaging can be modeled via the minimization of a nonconvex...
research
05/10/2013

Regularized M-estimators with nonconvexity: Statistical and algorithmic theory for local optima

We provide novel theoretical results regarding local optima of regulariz...
research
09/30/2014

Douglas-Rachford splitting for nonconvex optimization with application to nonconvex feasibility problems

We adapt the Douglas-Rachford (DR) splitting method to solve nonconvex f...
research
10/06/2019

Statistical Analysis of Stationary Solutions of Coupled Nonconvex Nonsmooth Empirical Risk Minimization

This paper has two main goals: (a) establish several statistical propert...

Please sign up or login with your details

Forgot password? Click here to reset