Distributed Contextual Linear Bandits with Minimax Optimal Communication Cost

05/26/2022
by   Sanae Amani, et al.
0

We study distributed contextual linear bandits with stochastic contexts, where N agents act cooperatively to solve a linear bandit-optimization problem with d-dimensional features. For this problem, we propose a distributed batch elimination version of the LinUCB algorithm, DisBE-LUCB, where the agents share information among each other through a central server. We prove that over T rounds (NT actions in total) the communication cost of DisBE-LUCB is only 𝒪̃(dN) and its regret is at most 𝒪̃(√(dNT)), which is of the same order as that incurred by an optimal single-agent algorithm for NT rounds. Remarkably, we derive an information-theoretic lower bound on the communication cost of the distributed contextual linear bandit problem with stochastic contexts, and prove that our proposed algorithm is nearly minimax optimal in terms of both regret and communication cost. Finally, we propose DecBE-LUCB, a fully decentralized version of DisBE-LUCB, which operates without a central server, where agents share information with their immediate neighbors through a carefully designed consensus procedure.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
12/01/2020

Decentralized Multi-Agent Linear Bandits with Safety Constraints

We study decentralized stochastic linear bandits, where a network of N a...
research
11/04/2022

Distributed Linear Bandits under Communication Constraints

We consider distributed linear bandits where M agents learn collaborativ...
research
03/29/2023

Federated Stochastic Bandit Learning with Unobserved Context

We study the problem of federated stochastic multi-arm contextual bandit...
research
07/07/2022

A Simple and Provably Efficient Algorithm for Asynchronous Federated Contextual Linear Bandits

We study federated contextual linear bandits, where M agents cooperate w...
research
07/28/2022

Distributed Stochastic Bandit Learning with Context Distributions

We study the problem of distributed stochastic multi-arm contextual band...
research
04/26/2016

Distributed Clustering of Linear Bandits in Peer to Peer Networks

We provide two distributed confidence ball algorithms for solving linear...
research
02/09/2021

A Multi-Arm Bandit Approach To Subset Selection Under Constraints

We explore the class of problems where a central planner needs to select...

Please sign up or login with your details

Forgot password? Click here to reset