Dualize, Split, Randomize: Fast Nonsmooth Optimization Algorithms

04/03/2020
by   Adil Salim, et al.
8

We introduce a new primal-dual algorithm for minimizing the sum of three convex functions, each of which has its own oracle. Namely, the first one is differentiable, smooth and possibly stochastic, the second is proximable, and the last one is a composition of a proximable function with a linear map. Our theory covers several settings that are not tackled by any existing algorithm; we illustrate their importance with real-world applications. By leveraging variance reduction, we obtain convergence with linear rates under strong convexity and fast sublinear convergence under convexity assumptions. The proposed theory is simple and unified by the umbrella of stochastic Davis-Yin splitting, which we design in this work. Finally, we illustrate the efficiency of our method through numerical experiments.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

05/28/2019

Stochastic Proximal Langevin Algorithm: Potential Splitting and Nonasymptotic Rates

We propose a new algorithm---Stochastic Proximal Langevin Algorithm (SPL...
12/22/2021

A Stochastic Bregman Primal-Dual Splitting Algorithm for Composite Optimization

We study a stochastic first order primal-dual method for solving convex-...
05/29/2019

A unified variance-reduced accelerated gradient method for convex optimization

We propose a novel randomized incremental gradient algorithm, namely, VA...
07/31/2015

Fast Stochastic Algorithms for SVD and PCA: Convergence Properties and Convexity

We study the convergence properties of the VR-PCA algorithm introduced b...
03/08/2022

Mini-batch stochastic three-operator splitting for distributed optimization

We consider a network of agents, each with its own private cost consisti...
01/24/2019

SAGA with Arbitrary Sampling

We study the problem of minimizing the average of a very large number of...
07/07/2014

The Primal-Dual Hybrid Gradient Method for Semiconvex Splittings

This paper deals with the analysis of a recent reformulation of the prim...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.