Adaptive Bandit Convex Optimization with Heterogeneous Curvature

02/12/2022
by   Haipeng Luo, et al.
0

We consider the problem of adversarial bandit convex optimization, that is, online learning over a sequence of arbitrary convex loss functions with only one function evaluation for each of them. While all previous works assume known and homogeneous curvature on these loss functions, we study a heterogeneous setting where each function has its own curvature that is only revealed after the learner makes a decision. We develop an efficient algorithm that is able to adapt to the curvature on the fly. Specifically, our algorithm not only recovers or even improves existing results for several homogeneous settings, but also leads to surprising results for some heterogeneous settings – for example, while Hazan and Levy (2014) showed that O(d^3/2√(T)) regret is achievable for a sequence of T smooth and strongly convex d-dimensional functions, our algorithm reveals that the same is achievable even if T^3/4 of them are not strongly convex, and sometimes even if a constant fraction of them are not strongly convex. Our approach is inspired by the framework of Bartlett et al. (2007) who studied a similar heterogeneous setting but with stronger gradient feedback. Extending their framework to the bandit feedback setting requires novel ideas such as lifting the feasible domain and using a logarithmically homogeneous self-concordant barrier regularizer.

READ FULL TEXT
research
07/01/2020

Bandit Linear Control

We consider the problem of controlling a known linear dynamical system u...
research
07/16/2020

Comparator-adaptive Convex Bandits

We study bandit convex optimization methods that adapt to the norm of th...
research
08/12/2020

Non-Stochastic Control with Bandit Feedback

We study the problem of controlling a linear dynamical system with adver...
research
02/08/2020

Curvature of Feasible Sets in Offline and Online Optimization

It is known that the curvature of the feasible set in convex optimizatio...
research
04/18/2019

Semi-bandit Optimization in the Dispersed Setting

In this work, we study the problem of online optimization of piecewise L...
research
09/22/2016

(Bandit) Convex Optimization with Biased Noisy Gradient Oracles

Algorithms for bandit convex optimization and online learning often rely...
research
02/22/2017

Fast Rates for Bandit Optimization with Upper-Confidence Frank-Wolfe

We consider the problem of bandit optimization, inspired by stochastic o...

Please sign up or login with your details

Forgot password? Click here to reset