On Adaptivity in Information-constrained Online Learning

10/19/2019
by   Siddharth Mitra, et al.
0

We study how to adapt to smoothly-varying (`easy') environments in well-known online learning problems where acquiring information is expensive. For the problem of label efficient prediction, which is a budgeted version of prediction with expert advice, we present an online algorithm whose regret depends optimally on the number of labels allowed and Q^* (the quadratic variation of the losses of the best action in hindsight), along with a parameter-free counterpart whose regret depends optimally on Q (the quadratic variation of the losses of all the actions). These quantities can be significantly smaller than T (the total time horizon), yielding an improvement over existing, variation-independent results for the problem. We then extend our analysis to handle label efficient prediction with bandit feedback, i.e., label efficient bandits. Our work builds upon the framework of optimistic online mirror descent, and leverages second order corrections along with a carefully designed hybrid regularizer that encodes the constrained information structure of the problem. We then consider revealing action-partial monitoring games – a version of label efficient prediction with additive information costs, which in general are known to lie in the hard class of games having minimax regret of order T^2/3. We provide a strategy with an O((Q^*T)^1/3) bound for revealing action games, along with an one with a O((QT)^1/3) bound for the full class of hard partial monitoring games, both being strict improvements over current bounds.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/13/2022

Second Order Regret Bounds Against Generalized Expert Sequences under Partial Bandit Feedback

We study the problem of expert advice under partial bandit feedback sett...
research
02/27/2017

Algorithmic Chaining and the Role of Partial Feedback in Online Nonparametric Learning

We investigate contextual online learning with nonparametric (Lipschitz)...
research
03/07/2020

Adversarial Online Learning with Changing Action Sets: Efficient Algorithms with Approximate Regret Bounds

We revisit the problem of online learning with sleeping experts/bandits:...
research
06/18/2022

Optimal Dynamic Regret in LQR Control

We consider the problem of nonstochastic control with a sequence of quad...
research
09/30/2014

Nonstochastic Multi-Armed Bandits with Graph-Structured Feedback

We present and study a partial-information model of online learning, whe...
research
06/05/2023

Online Learning with Feedback Graphs: The True Shape of Regret

Sequential learning with feedback graphs is a natural extension of the m...
research
02/16/2023

Adaptive Selective Sampling for Online Prediction with Experts

We consider online prediction of a binary sequence with expert advice. F...

Please sign up or login with your details

Forgot password? Click here to reset