Faster Projection-free Online Learning

01/30/2020
by   Elad Hazan, et al.
0

In many online learning problems the computational bottleneck for gradient-based methods is the projection operation. For this reason, in many problems the most efficient algorithms are based on the Frank-Wolfe method, which replaces projections by linear optimization. In the general case, however, online projection-free methods require more iterations than projection-based methods: the best known regret bound scales as T^3/4. Despite significant work on various variants of the Frank-Wolfe method, this bound has remained unchanged for a decade. In this paper we give an efficient projection-free algorithm that guarantees T^2/3 regret for general online convex optimization with smooth cost functions and one linear optimization computation per iteration. As opposed to previous Frank-Wolfe approaches, our algorithm is derived using the Follow-the-Perturbed-Leader method and is analyzed using an online primal-dual framework.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/08/2019

Improved Regret Bounds for Projection-free Bandit Convex Optimization

We revisit the challenge of designing online algorithms for the bandit c...
research
02/05/2018

Online Compact Convexified Factorization Machine

Factorization Machine (FM) is a supervised learning approach with a powe...
research
10/21/2019

Stochastic Recursive Gradient-Based Methods for Projection-Free Online Learning

This paper focuses on projection-free methods for solving smooth Online ...
research
11/22/2022

Projection-free Adaptive Regret with Membership Oracles

In the framework of online convex optimization, most iterative algorithm...
research
10/21/2019

Efficient Projection-Free Online Methods with Stochastic Recursive Gradient

This paper focuses on projection-free methods for solving smooth Online ...
research
09/13/2023

A Flexible Online Framework for Projection-Based STFT Phase Retrieval

Several recent contributions in the field of iterative STFT phase retrie...
research
10/15/2020

Revisiting Projection-free Online Learning: the Strongly Convex Case

Projection-free optimization algorithms, which are mostly based on the c...

Please sign up or login with your details

Forgot password? Click here to reset