An Efficient Algorithm for Cooperative Semi-Bandits

10/05/2020
by   Riccardo Della Vecchia, et al.
0

We consider the problem of asynchronous online combinatorial optimization on a network of communicating agents. At each time step, some of the agents are stochastically activated, requested to make a prediction, and the system pays the corresponding loss. Then, neighbors of active agents receive semi-bandit feedback and exchange some succinct local information. The goal is to minimize the network regret, defined as the difference between the cumulative loss of the predictions of active agents and that of the best action in hindsight, selected from a combinatorial decision set. The main challenge in such a context is to control the computational complexity of the resulting algorithm while retaining minimax optimal regret guarantees. We introduce Coop-FTPL, a cooperative version of the well-known Follow The Perturbed Leader algorithm, that implements a new loss estimation procedure generalizing the Geometric Resampling of Neu and Bartók [2013] to our setting. Assuming that the elements of the decision set are k-dimensional binary vectors with at most m non-zero entries and α_1 is the independence number of the network, we show that the expected regret of our algorithm after T time steps is of order Q√(mkTlog(k) (kα_1/Q+m)), where Q is the total activation probability mass. Furthermore, we prove that this is only √(klog k)-away from the best achievable rate and that has a state-of-the-art T^3/2 worst-case computational complexity.

READ FULL TEXT

page 1

page 2

page 3

page 4

02/23/2015

First-order regret bounds for combinatorial semi-bandits

We consider the problem of online combinatorial optimization under semi-...
04/20/2012

Regret in Online Combinatorial Optimization

We address online linear optimization problems when the possible actions...
03/17/2015

Importance weighting without importance weights: An efficient algorithm for combinatorial semi-bandits

We propose a sample-efficient alternative for importance weighting for s...
06/09/2021

Cooperative Online Learning

In this preliminary (and unpolished) version of the paper, we study an a...
01/23/2019

Cooperative Online Learning: Keeping your Neighbors Updated

We study an asynchronous online learning setting with a network of agent...
02/17/2020

Statistically Efficient, Polynomial Time Algorithms for Combinatorial Semi Bandits

We consider combinatorial semi-bandits over a set of arms X⊂{0,1}^d wher...
10/09/2019

Robust Monopoly Regulation

We study the regulation of a monopolistic firm using a robust-design app...