Inducing Causal Structure for Interpretable Neural Networks

12/01/2021
by   Atticus Geiger, et al.
0

In many areas, we have well-founded insights about causal structure that would be useful to bring into our trained models while still allowing them to learn in a data-driven fashion. To achieve this, we present the new method of interchange intervention training(IIT). In IIT, we (1)align variables in the causal model with representations in the neural model and (2) train a neural model to match the counterfactual behavior of the causal model on a base input when aligned representations in both models are set to be the value they would be for a second source input. IIT is fully differentiable, flexibly combines with other objectives, and guarantees that the target causal model is acausal abstraction of the neural model when its loss is minimized. We evaluate IIT on a structured vision task (MNIST-PVR) and a navigational instruction task (ReaSCAN). We compare IIT against multi-task training objectives and data augmentation. In all our experiments, IIT achieves the best results and produces neural models that are more interpretable in the sense that they realize the target causal model.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 6

06/06/2021

Causal Abstractions of Neural Networks

Structural analysis methods (e.g., probing and feature attribution) are ...
06/03/2020

Explaining The Behavior Of Black-Box Prediction Algorithms With Causal Learning

We propose to explain the behavior of black-box prediction methods (e.g....
08/21/2020

Amortized learning of neural causal representations

Causal models can compactly and efficiently encode the data-generating p...
11/24/2017

Causal Generative Neural Networks

We introduce CGNN, a framework to learn functional causal models as gene...
05/03/2020

A Causal View on Robustness of Neural Networks

We present a causal view on the robustness of neural networks against in...
08/21/2021

Learning Causal Models of Autonomous Agents using Interventions

One of the several obstacles in the widespread use of AI systems is the ...
05/20/2019

Interpretable Neural Predictions with Differentiable Binary Variables

The success of neural networks comes hand in hand with a desire for more...

Code Repositories

ReaSCAN-Causal

Causal Abstraction of Neural Models Trained to Solve ReaSCAN


view repo

Interchange-Intervention-Training

None


view repo
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.