A single-phase, proximal path-following framework

03/05/2016
by   Quoc Tran-Dinh, et al.
0

We propose a new proximal, path-following framework for a class of constrained convex problems. We consider settings where the nonlinear---and possibly non-smooth---objective part is endowed with a proximity operator, and the constraint set is equipped with a self-concordant barrier. Our approach relies on the following two main ideas. First, we re-parameterize the optimality condition as an auxiliary problem, such that a good initial point is available; by doing so, a family of alternative paths towards the optimum is generated. Second, we combine the proximal operator with path-following ideas to design a single-phase, proximal, path-following algorithm. Our method has several advantages. First, it allows handling non-smooth objectives via proximal operators; this avoids lifting the problem dimension in order to accommodate non-smooth components in optimization. Second, it consists of only a single phase: While the overall convergence rate of classical path-following schemes for self-concordant objectives does not suffer from the initialization phase, proximal path-following schemes undergo slow convergence, in order to obtain a good starting point TranDinh2013e. In this work, we show how to overcome this limitation in the proximal setting and prove that our scheme has the same O(√(ν)(1/ε)) worst-case iteration-complexity with standard approaches Nesterov2004,Nesterov1994 without requiring an initial phase, where ν is the barrier parameter and ε is a desired accuracy. Finally, our framework allows errors in the calculation of proximal-Newton directions, without sacrificing the worst-case iteration complexity. We demonstrate the merits of our algorithm via three numerical examples, where proximal operators play a key role.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
01/31/2016

A Proximal Stochastic Quasi-Newton Algorithm

In this paper, we discuss the problem of minimizing the sum of two conve...
research
02/12/2021

Proximal and Federated Random Reshuffling

Random Reshuffling (RR), also known as Stochastic Gradient Descent (SGD)...
research
07/01/2016

Randomized block proximal damped Newton method for composite self-concordant minimization

In this paper we consider the composite self-concordant (CSC) minimizati...
research
05/13/2014

Scalable sparse covariance estimation via self-concordance

We consider the class of convex minimization problems, composed of a sel...
research
05/31/2016

Iterative Smoothing Proximal Gradient for Regression with Structured Sparsity

In the context high-dimensionnal predictive models, we consider the prob...
research
04/06/2022

Unconstrained Proximal Operator: the Optimal Parameter for the Douglas-Rachford Type Primal-Dual Methods

In this work, we propose an alternative parametrized form of the proxima...
research
06/10/2020

Principled Analyses and Design of First-Order Methods with Inexact Proximal Operators

Proximal operations are among the most common primitives appearing in bo...

Please sign up or login with your details

Forgot password? Click here to reset