DeepAI AI Chat
Log In Sign Up

Dualize, Split, Randomize: Fast Nonsmooth Optimization Algorithms

by   Adil Salim, et al.

We introduce a new primal-dual algorithm for minimizing the sum of three convex functions, each of which has its own oracle. Namely, the first one is differentiable, smooth and possibly stochastic, the second is proximable, and the last one is a composition of a proximable function with a linear map. Our theory covers several settings that are not tackled by any existing algorithm; we illustrate their importance with real-world applications. By leveraging variance reduction, we obtain convergence with linear rates under strong convexity and fast sublinear convergence under convexity assumptions. The proposed theory is simple and unified by the umbrella of stochastic Davis-Yin splitting, which we design in this work. Finally, we illustrate the efficiency of our method through numerical experiments.


page 1

page 2

page 3

page 4


DISA: A Dual Inexact Splitting Algorithm for Distributed Convex Composite Optimization

This paper proposes a novel dual inexact splitting algorithm (DISA) for ...

Fast Stochastic Composite Minimization and an Accelerated Frank-Wolfe Algorithm under Parallelization

We consider the problem of minimizing the sum of two convex functions. O...

Stochastic Proximal Langevin Algorithm: Potential Splitting and Nonasymptotic Rates

We propose a new algorithm---Stochastic Proximal Langevin Algorithm (SPL...

A Stochastic Bregman Primal-Dual Splitting Algorithm for Composite Optimization

We study a stochastic first order primal-dual method for solving convex-...

Fast Stochastic Algorithms for SVD and PCA: Convergence Properties and Convexity

We study the convergence properties of the VR-PCA algorithm introduced b...

Mini-batch stochastic three-operator splitting for distributed optimization

We consider a network of agents, each with its own private cost consisti...

SAGA with Arbitrary Sampling

We study the problem of minimizing the average of a very large number of...