Minding the Gaps for Block Frank-Wolfe Optimization of Structured SVMs

05/30/2016
by   Anton Osokin, et al.
0

In this paper, we propose several improvements on the block-coordinate Frank-Wolfe (BCFW) algorithm from Lacoste-Julien et al. (2013) recently used to optimize the structured support vector machine (SSVM) objective in the context of structured prediction, though it has wider applications. The key intuition behind our improvements is that the estimates of block gaps maintained by BCFW reveal the block suboptimality that can be used as an adaptive criterion. First, we sample objects at each iteration of BCFW in an adaptive non-uniform way via gapbased sampling. Second, we incorporate pairwise and away-step variants of Frank-Wolfe into the block-coordinate setting. Third, we cache oracle calls with a cache-hit criterion based on the block gaps. Fourth, we provide the first method to compute an approximate regularization path for SSVM. Finally, we provide an exhaustive empirical evaluation of all our methods on four structured prediction datasets.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/19/2012

Block-Coordinate Frank-Wolfe Optimization for Structural SVMs

We propose a randomized block-coordinate variant of the classic Frank-Wo...
research
06/08/2015

Distributed Training of Structured SVM

Training structured prediction models is time-consuming. However, most e...
research
11/21/2018

Multi-Block Grid deformation Method in 3D

Multi-block grids provide the computational efficiency of structured gri...
research
01/15/2014

Coordinate Descent with Online Adaptation of Coordinate Frequencies

Coordinate descent (CD) algorithms have become the method of choice for ...
research
10/21/2019

Relative Interior Rule in Block-Coordinate Minimization

(Block-)coordinate minimization is an iterative optimization method whic...
research
01/13/2020

Accelerating Block Coordinate Descent for Nonnegative Tensor Factorization

This paper is concerned with improving the empirical convergence speed o...
research
10/08/2020

A User-Friendly Computational Framework for Robust Structured Regression Using the L_2 Criterion

We introduce a user-friendly computational framework for implementing ro...

Please sign up or login with your details

Forgot password? Click here to reset