Faster One-Sample Stochastic Conditional Gradient Method for Composite Convex Minimization

02/26/2022
by   Gideon Dresdner, et al.
0

We propose a stochastic conditional gradient method (CGM) for minimizing convex finite-sum objectives formed as a sum of smooth and non-smooth terms. Existing CGM variants for this template either suffer from slow convergence rates, or require carefully increasing the batch size over the course of the algorithm's execution, which leads to computing full gradients. In contrast, the proposed method, equipped with a stochastic average gradient (SAG) estimator, requires only one sample per iteration. Nevertheless, it guarantees fast convergence rates on par with more sophisticated variance reduction techniques. In applications we put special emphasis on problems with a large number of separable constraints. Such problems are prevalent among semidefinite programming (SDP) formulations arising in machine learning and theoretical computer science. We provide numerical experiments on matrix completion, unsupervised clustering, and sparsest-cut SDPs.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/07/2020

Conditional gradient methods for stochastically constrained convex minimization

We propose two novel conditional gradient-based methods for solving stru...
research
04/23/2023

Sarah Frank-Wolfe: Methods for Constrained Optimization with Best Rates and Practical Features

The Frank-Wolfe (FW) method is a popular approach for solving optimizati...
research
01/29/2019

Stochastic Conditional Gradient Method for Composite Convex Minimization

In this paper, we propose the first practical algorithm to minimize stoc...
research
09/10/2013

Minimizing Finite Sums with the Stochastic Average Gradient

We propose the stochastic average gradient (SAG) method for optimizing t...
research
04/13/2011

Hybrid Deterministic-Stochastic Methods for Data Fitting

Many structured data-fitting applications require the solution of an opt...
research
07/07/2022

A conditional gradient homotopy method with applications to Semidefinite Programming

We propose a new homotopy-based conditional gradient method for solving ...
research
06/22/2021

Reusing Combinatorial Structure: Faster Iterative Projections over Submodular Base Polytopes

Optimization algorithms such as projected Newton's method, FISTA, mirror...

Please sign up or login with your details

Forgot password? Click here to reset