Follow the Compressed Leader: Faster Online Learning of Eigenvectors and Faster MMWU

01/06/2017
by   Zeyuan Allen-Zhu, et al.
0

The online problem of computing the top eigenvector is fundamental to machine learning. In both adversarial and stochastic settings, previous results (such as matrix multiplicative weight update, follow the regularized leader, follow the compressed leader, block power method) either achieve optimal regret but run slow, or run fast at the expense of loosing a √(d) factor in total regret where d is the matrix dimension. We propose a follow-the-compressed-leader (FTCL) framework which achieves optimal regret without sacrificing the running time. Our idea is to "compress" the matrix strategy to dimension 3 in the adversarial setting, or dimension 1 in the stochastic setting. These respectively resolve two open questions regarding the design of optimal and efficient algorithms for the online eigenvector problem.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/19/2019

Online Non-Convex Learning: Following the Perturbed Leader is Optimal

We study the problem of online learning with non-convex losses, where th...
research
06/16/2015

Spectral Sparsification and Regret Minimization Beyond Matrix Multiplicative Updates

In this paper, we provide a novel construction of the linear-sized spect...
research
02/06/2022

Pushing the Efficiency-Regret Pareto Frontier for Online Learning of Portfolios and Quantum States

We revisit the classical online portfolio selection problem. It is widel...
research
04/04/2012

Relax and Localize: From Value to Algorithms

We show a principled way of deriving online learning algorithms from a m...
research
06/16/2015

PCA with Gaussian perturbations

Most of machine learning deals with vector parameters. Ideally we would ...
research
04/03/2021

Modeling and Development of Operation Guidelines for Leader-Follower Autonomous Truck-Mounted Attenuator Vehicles

Mobile and slow moving operations, such as striping, sweeping, bridge fl...
research
02/24/2020

Prediction with Corrupted Expert Advice

We revisit the fundamental problem of prediction with expert advice, in ...

Please sign up or login with your details

Forgot password? Click here to reset