DeepAI AI Chat
Log In Sign Up

A primal-dual algorithm with optimal stepsizes and its application in decentralized consensus optimization

by   Zhi Li, et al.
Michigan State University

We consider a primal-dual algorithm for minimizing f(x)+h(Ax) with differentiable f. The primal-dual algorithm has two names in literature: Primal-Dual Fixed-Point algorithm based on the Proximity Operator (PDFP^2O) and Proximal Alternating Predictor-Corrector (PAPC). In this paper, we extend it to solve f(x)+h l(Ax) with differentiable l^* and prove its convergence under a weak condition (i.e., under a large dual stepsize). With additional assumptions, we show its linear convergence. In addition, we show that this condition is optimal and can not be weaken. This result recovers the recent proposed positive-indefinite linearized augmented Lagrangian method. Then we consider the application of this primal-dual algorithm in decentralized consensus optimization. We show that EXact firsT-ordeR Algorithm (EXTRA) and Proximal Gradient-EXTRA (PG-EXTRA) can be consider as the primal-dual algorithm applied on a problem in the form of h l(Ax). Then, the optimal upper bound of the stepsize for EXTRA/PG-EXTRA is derived. It is larger than the existing work on EXTRA/PG-EXTRA. Furthermore, for the case with strongly convex functions, we proved linear convergence under the same condition for the stepsize.


page 1

page 2

page 3

page 4


On the improved conditions for some primal-dual algorithms

The convex minimization of f(𝐱)+g(𝐱)+h(𝐀𝐱) over ℝ^n with differentiable ...

A Riemannian Primal-dual Algorithm Based on Proximal Operator and its Application in Metric Learning

In this paper, we consider optimizing a smooth, convex, lower semicontin...

Chambolle-Pock's Primal-Dual Method with Mismatched Adjoint

The primal-dual method of Chambolle and Pock is a widely used algorithm ...

IDEAL: Inexact DEcentralized Accelerated Augmented Lagrangian Method

We introduce a framework for designing primal methods under the decentra...

On linear convergence of two decentralized algorithms

Decentralized algorithms solve multi-agent problems over a connected net...

Approximate Primal-Dual Fixed-Point based Langevin Algorithms for Non-smooth Convex Potentials

The Langevin algorithms are frequently used to sample the posterior dist...

Unconstrained Proximal Operator: the Optimal Parameter for the Douglas-Rachford Type Primal-Dual Methods

In this work, we propose an alternative parametrized form of the proxima...