Revisiting EXTRA for Smooth Distributed Optimization

02/24/2020
by   Huan Li, et al.
0

EXTRA is a popular method for the dencentralized distributed optimization and has broad applications. This paper revisits the EXTRA. Firstly, we give a sharp complexity analysis for EXTRA with the improved O((L/μ+1/1-σ_2(W))log1/ϵ(1-σ_2(W))) communication and computation complexities for μ-strongly convex and L-smooth problems, where σ_2(W) is the second largest singular value of the weight matrix W. When the strong convexity is absent, we prove the O((L/ϵ+1/1-σ_2(W))log1/1-σ_2(W)) complexities. Then, we use the Catalyst framework to accelerate EXTRA and obtain the O(√(L/μ(1-σ_2(W)))log L/μ(1-σ_2(W))log1/ϵ) communication and computation complexities for strongly convex and smooth problems and the O(√(L/ϵ(1-σ_2(W)))log1/ϵ(1-σ_2(W))) complexities for non-strongly convex ones. Our communication complexities of the accelerated EXTRA are only worse by the factors of (logL/μ(1-σ_2(W))) and (log1/ϵ(1-σ_2(W))) from the lower complexity bounds for strongly convex and non-strongly convex problems, respectively.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
09/09/2020

Variance Reduced EXTRA and DIGing and Their Optimal Acceleration for Strongly Convex Decentralized Optimization

We study stochastic decentralized optimization for the problem of traini...
research
04/06/2021

Accelerated Gradient Tracking over Time-varying Graphs for Decentralized Optimization

Decentralized optimization over time-varying graphs has been increasingl...
research
02/07/2021

Lower Bounds and Accelerated Algorithms for Bilevel Optimization

Bilevel optimization has recently attracted growing interests due to its...
research
12/01/2017

Optimal Algorithms for Distributed Optimization

In this paper, we study the optimal convergence rate for distributed con...
research
02/15/2022

Optimal Algorithms for Stochastic Multi-Level Compositional Optimization

In this paper, we investigate the problem of stochastic multi-level comp...
research
03/02/2020

Smooth Strongly Convex Regression

Convex regression (CR) is the problem of fitting a convex function to a ...
research
01/28/2019

Space and complexities of territorial systems

The spatial character of territorial systems plays a crucial role in the...

Please sign up or login with your details

Forgot password? Click here to reset