DeepAI AI Chat
Log In Sign Up

On the Sample Complexity of Representation Learning in Multi-task Bandits with Global and Local structure

by   Alessio Russo, et al.

We investigate the sample complexity of learning the optimal arm for multi-task bandit problems. Arms consist of two components: one that is shared across tasks (that we call representation) and one that is task-specific (that we call predictor). The objective is to learn the optimal (representation, predictor)-pair for each task, under the assumption that the optimal representation is common to all tasks. Within this framework, efficient learning algorithms should transfer knowledge across tasks. We consider the best-arm identification problem for a fixed confidence, where, in each round, the learner actively selects both a task, and an arm, and observes the corresponding reward. We derive instance-specific sample complexity lower bounds satisfied by any (δ_G,δ_H)-PAC algorithm (such an algorithm identifies the best representation with probability at least 1-δ_G, and the best predictor for a task with probability at least 1-δ_H). We devise an algorithm OSRL-SC whose sample complexity approaches the lower bound, and scales at most as H(Glog(1/δ_G)+ Xlog(1/δ_H)), with X,G,H being, respectively, the number of tasks, representations and predictors. By comparison, this scaling is significantly better than the classical best-arm identification algorithm that scales as HGXlog(1/δ).


page 1

page 2

page 3

page 4


Multi-task Representation Learning for Pure Exploration in Linear Bandits

Despite the recent success of representation learning in sequential deci...

On the Theory of Transfer Learning: The Importance of Task Diversity

We provide new statistical guarantees for transfer learning via represen...

Evaluating representations by the complexity of learning low-loss predictors

We consider the problem of evaluating representations of data for use in...

Best-arm identification with cascading bandits

We consider a variant of the problem of best arm identification in multi...

Best arm identification in rare events

We consider the best arm identification problem in the stochastic multi-...

Best Arm Identification in Spectral Bandits

We study best-arm identification with fixed confidence in bandit models ...

Interactively Learning Preference Constraints in Linear Bandits

We study sequential decision-making with known rewards and unknown const...