Causal Bandits for Linear Structural Equation Models

08/26/2022
by   Burak Varici, et al.
0

This paper studies the problem of designing an optimal sequence of interventions in a causal graphical model to minimize the cumulative regret with respect to the best intervention in hindsight. This is, naturally, posed as a causal bandit problem. The focus is on causal bandits for linear structural equation models (SEMs) and soft interventions. It is assumed that the graph's structure is known, and it has N nodes. Two linear mechanisms, one soft intervention and one observational, are assumed for each node, giving rise to 2^N possible interventions. The existing causal bandit algorithms assume that at least the interventional distributions of the reward node's parents are fully specified. However, there are 2^N such distributions (one corresponding to each intervention), acquiring which becomes prohibitive even in moderate-sized graphs. This paper dispenses with the assumption of knowing these distributions. Two algorithms are proposed for the frequentist (UCB-based) and Bayesian (Thompson Sampling-based) settings. The key idea of these algorithms is to avoid directly estimating the 2^N reward distributions and instead estimate the parameters that fully specify the SEMs (linear in N) and use them to compute the rewards. In both algorithms, under boundedness assumptions on noise and the parameter space, the cumulative regrets scale as Õ ((2d)^L L √(T)), where d is the graph's maximum degree, and L is the length of its longest causal path.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
12/13/2020

Budgeted and Non-budgeted Causal Bandits

Learning good interventions in a causal graph can be modelled as a stoch...
research
06/06/2018

Causal Bandits with Propagating Inference

Bandit is a framework for designing sequential experiments. In each expe...
research
10/11/2019

Regret Analysis of Causal Bandit Problems

We study how to learn optimal interventions sequentially given causal in...
research
07/31/2023

A continuous Structural Intervention Distance to compare Causal Graphs

Understanding and adequately assessing the difference between a true and...
research
06/05/2021

Causal Bandits with Unknown Graph Structure

In causal bandit problems, the action set consists of interventions on v...
research
01/10/2017

Identifying Best Interventions through Online Importance Sampling

Motivated by applications in computational advertising and systems biolo...
research
01/31/2023

Combinatorial Causal Bandits without Graph Skeleton

In combinatorial causal bandits (CCB), the learning agent chooses a subs...

Please sign up or login with your details

Forgot password? Click here to reset