DeepAI AI Chat
Log In Sign Up

Blended Conditional Gradients: the unconditioning of conditional gradients

by   Gabor Braun, et al.
University of Wisconsin-Madison
Georgia Institute of Technology

We present a blended conditional gradient approach for minimizing a smooth convex function over a polytope P, that combines the Frank--Wolfe algorithm (also called conditional gradient) with gradient-based steps different from away steps and pairwise steps, however, still achieving linear convergence for strongly convex functions and good practical performance. Our approach retains all favorable properties of conditional gradient algorithms, most notably avoidance of projections onto P and maintenance of iterates as sparse convex combinations of a limited number of extreme points of P. The algorithm decreases measures of optimality (primal and dual gaps) rapidly, both in the number of iterations and in wall-clock time, outperforming even the efficient "lazified" conditional gradient algorithms of [arXiv:1410.8816]. Nota bene the algorithm is lazified itself. We also present a streamlined algorithm when P is the probability simplex.


page 1

page 2

page 3

page 4


Locally Accelerated Conditional Gradients

Conditional gradient methods form a class of projection-free first-order...

Walking in the Shadow: A New Perspective on Descent Directions for Constrained Minimization

Descent directions such as movement towards Frank-Wolfe vertices, away s...

A first-order primal-dual method with adaptivity to local smoothness

We consider the problem of finding a saddle point for the convex-concave...

Practical Frank-Wolfe algorithms

In the last decade there has been a resurgence of interest in Frank-Wolf...

Blended Matching Pursuit

Matching pursuit algorithms are an important class of algorithms in sign...

Conditional gradient methods for stochastically constrained convex minimization

We propose two novel conditional gradient-based methods for solving stru...