Linear Bandit Algorithms with Sublinear Time Complexity

03/03/2021
by   Shuo Yang, et al.
18

We propose to accelerate existing linear bandit algorithms to achieve per-step time complexity sublinear in the number of arms K. The key to sublinear complexity is the realization that the arm selection in many linear bandit algorithms reduces to the maximum inner product search (MIPS) problem. Correspondingly, we propose an algorithm that approximately solves the MIPS problem for a sequence of adaptive queries yielding near-linear preprocessing time complexity and sublinear query time complexity. Using the proposed MIPS solver as a sub-routine, we present two bandit algorithms (one based on UCB, and the other based on TS) that achieve sublinear time complexity. We explicitly characterize the tradeoff between the per-step time complexity and regret, and show that our proposed algorithms can achieve O(K^1-α(T)) per-step complexity for some α(T) > 0 and O(√(T)) regret, where T is the time horizon. Further, we present the theoretical limit of the tradeoff, which provides a lower bound for the per-step time complexity. We also discuss other choices of approximate MIPS algorithms and other applications to linear bandit problems.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/01/2017

Scalable Generalized Linear Bandits: Online Computation and Hashing

Generalized Linear Bandits (GLBs), a natural extension of the stochastic...
research
12/15/2018

A Bandit Approach to Maximum Inner Product Search

There has been substantial research on sub-linear time approximate algor...
research
02/26/2020

Memory-Constrained No-Regret Learning in Adversarial Bandits

An adversarial bandit problem with memory constraints is studied where o...
research
05/18/2021

Sublinear Least-Squares Value Iteration via Locality Sensitive Hashing

We present the first provable Least-Squares Value Iteration (LSVI) algor...
research
11/13/2019

Context-aware Dynamic Assets Selection for Online Portfolio Selection based on Contextual Bandit

Online portfolio selection is a sequential decision-making problem in fi...
research
02/17/2022

Fast online inference for nonlinear contextual bandit based on Generative Adversarial Network

This work addresses the efficiency concern on inferring a nonlinear cont...
research
05/09/2020

Time complexity of in-memory solution of linear systems

In-memory computing with crosspoint resistive memory arrays has been sho...

Please sign up or login with your details

Forgot password? Click here to reset