DeepAI AI Chat
Log In Sign Up

Inducing Causal Structure for Interpretable Neural Networks

by   Atticus Geiger, et al.

In many areas, we have well-founded insights about causal structure that would be useful to bring into our trained models while still allowing them to learn in a data-driven fashion. To achieve this, we present the new method of interchange intervention training(IIT). In IIT, we (1)align variables in the causal model with representations in the neural model and (2) train a neural model to match the counterfactual behavior of the causal model on a base input when aligned representations in both models are set to be the value they would be for a second source input. IIT is fully differentiable, flexibly combines with other objectives, and guarantees that the target causal model is acausal abstraction of the neural model when its loss is minimized. We evaluate IIT on a structured vision task (MNIST-PVR) and a navigational instruction task (ReaSCAN). We compare IIT against multi-task training objectives and data augmentation. In all our experiments, IIT achieves the best results and produces neural models that are more interpretable in the sense that they realize the target causal model.


Causal Abstractions of Neural Networks

Structural analysis methods (e.g., probing and feature attribution) are ...

Explaining The Behavior Of Black-Box Prediction Algorithms With Causal Learning

We propose to explain the behavior of black-box prediction methods (e.g....

Naturalistic Causal Probing for Morpho-Syntax

Probing has become a go-to methodology for interpreting and analyzing de...

Finding Alignments Between Interpretable Causal Variables and Distributed Neural Representations

Causal abstraction is a promising theoretical framework for explainable ...

Amortized learning of neural causal representations

Causal models can compactly and efficiently encode the data-generating p...

A Causal View on Robustness of Neural Networks

We present a causal view on the robustness of neural networks against in...

Learning Causal Models of Autonomous Agents using Interventions

One of the several obstacles in the widespread use of AI systems is the ...

Code Repositories


Causal Abstraction of Neural Models Trained to Solve ReaSCAN

view repo



view repo