Counterfactual Data Augmentation using Locally Factored Dynamics

by   Silviu Pitis, et al.

Many dynamic processes, including common scenarios in robotic control and reinforcement learning (RL), involve a set of interacting subprocesses. Though the subprocesses are not independent, their interactions are often sparse, and the dynamics at any given time step can often be decomposed into locally independent causal mechanisms. Such local causal structures can be leveraged to improve the sample efficiency of sequence prediction and off-policy reinforcement learning. We formalize this by introducing local causal models (LCMs), which are induced from a global causal model by conditioning on a subset of the state space. We propose an approach to inferring these structures given an object-oriented state representation, as well as a novel algorithm for model-free Counterfactual Data Augmentation (CoDA). CoDA uses local structures and an experience replay to generate counterfactual experiences that are causally valid in the global model. We find that CoDA significantly improves the performance of RL agents in locally factored tasks, including the batch-constrained and goal-conditioned settings.


page 2

page 23


MoCoDA: Model-based Counterfactual Data Augmentation

The number of states in a dynamic process is exponential in the number o...

Sample-Efficient Reinforcement Learning via Counterfactual-Based Data Augmentation

Reinforcement learning (RL) algorithms usually require a substantial amo...

Policy Continuation with Hindsight Inverse Dynamics

Solving goal-oriented tasks is an important but challenging problem in r...

Causal Reasoning from Meta-reinforcement Learning

Discovering and exploiting the causal structure in the environment is a ...

Implicit Counterfactual Data Augmentation for Deep Neural Networks

Machine-learning models are prone to capturing the spurious correlations...

Counterfactual Analysis in Dynamic Models: Copulas and Bounds

We provide an explicit model of the causal mechanism in a structural cau...

Causal Discovery in Physical Systems from Videos

Causal discovery is at the core of human cognition. It enables us to rea...

Please sign up or login with your details

Forgot password? Click here to reset