
-
AI-SARAH: Adaptive and Implicit Stochastic Recursive Gradient Methods
We present an adaptive stochastic variance reduced method with an implic...
read it
-
Stochastic Hamiltonian Gradient Methods for Smooth Games
The success of adversarial formulations in machine learning has brought ...
read it
-
Unified Analysis of Stochastic Gradient Methods for Composite Convex and Smooth Optimization
We present a unified theorem for the convergence analysis of stochastic ...
read it
-
SGD for Structured Nonconvex Functions: Learning Rates, Minibatching and Interpolation
We provide several convergence theorems for SGD for two large classes of...
read it
-
A Unified Theory of Decentralized SGD with Changing Topology and Local Updates
Decentralized stochastic optimization methods have gained a lot of atten...
read it
-
Stochastic Polyak Step-size for SGD: An Adaptive Learning Rate for Fast Convergence
We propose a stochastic variant of the classical Polyak step-size (Polya...
read it
-
Randomized Iterative Methods for Linear Systems: Momentum, Inexactness and Gossip
In the era of big data, one of the key challenges is the development of ...
read it
-
Revisiting Randomized Gossip Algorithms: General Framework, Convergence Rates and Novel Block and Accelerated Protocols
In this work we present a new framework for the analysis and design of r...
read it
-
Convergence Analysis of Inexact Randomized Iterative Methods
In this paper we present a convergence rate analysis of inexact variants...
read it
-
SGD: General Analysis and Improved Rates
We propose a general yet simple theorem describing the convergence of SG...
read it
-
A Privacy Preserving Randomized Gossip Algorithm via Controlled Noise Insertion
In this work we present a randomized gossip algorithm for solving the av...
read it
-
Stochastic Gradient Push for Distributed Deep Learning
Large mini-batch parallel SGD is commonly used for distributed training ...
read it
-
Provably Accelerated Randomized Gossip Algorithms
In this work we present novel provably accelerated gossip algorithms for...
read it
-
Accelerated Gossip via Stochastic Heavy Ball Method
In this paper we show how the stochastic heavy ball method (SHB) -- a po...
read it
-
Momentum and Stochastic Momentum for Stochastic Gradient, Newton, Proximal Point and Subspace Descent Methods
In this paper we study several classes of stochastic optimization algori...
read it
-
Linearly convergent stochastic heavy ball method for minimizing generalization error
In this work we establish the first linear convergence result for the st...
read it