
Curvatureaided Incremental Aggregated Gradient Method
We propose a new algorithm for finite sum optimization which we call the...
read it

Minimizing Finite Sums with the Stochastic Average Gradient
We propose the stochastic average gradient (SAG) method for optimizing t...
read it

On Curvatureaided Incremental Aggregated Gradient Methods
This paper studies an acceleration technique for incremental aggregated ...
read it

Katyusha Acceleration for Convex FiniteSum Compositional Optimization
Structured problems arise in many applications. To solve these problems,...
read it

ASYSONATA: Achieving Geometric Convergence for Distributed Asynchronous Optimization
Can one obtain a geometrically convergent algorithm for distributed asyn...
read it

Convergence rate analysis and improved iterations for numerical radius computation
We analyze existing methods for computing the numerical radius and intro...
read it

Walkman: A CommunicationEfficient RandomWalk Algorithm for Decentralized Optimization
This paper addresses consensus optimization problems in a multiagent ne...
read it
SUCAG: Stochastic Unbiased Curvatureaided Gradient Method for Distributed Optimization
We propose and analyze a new stochastic gradient method, which we call Stochastic Unbiased Curvatureaided Gra dient (SUCAG), for finite sum optimization problems. SUCAG constitutes an unbiased total gradient tracking technique that uses Hessian information to accelerate convergence. We an alyze our method under the general asynchronous model of computation, in which functions are selected infinitely often, but with delays that can grow sublinearly. For strongly convex problems, we establish linear convergence for the SUCAG method. When the initialization point is sufficiently close to the optimal solution, the established convergence rate is only dependent on the condition number of the problem, making it strictly faster than the known rate for the SAGA method. Furthermore, we describe a Markovdriven approach of implementing the SUCAG method in a distributed asynchronous multiagent setting, via gossiping along a random walk on the communication graph. We show that our analysis applies as long as the undirected graph is connected and, notably, establishes an asymptotic linear convergence rate that is robust to the graph topology. Numerical results demonstrate the merit of our algorithm over existing methods.
READ FULL TEXT
Comments
There are no comments yet.