Log In Sign Up

CASTLE: Regularization via Auxiliary Causal Graph Discovery

by   Trent Kyono, et al.

Regularization improves generalization of supervised models to out-of-sample data. Prior works have shown that prediction in the causal direction (effect from cause) results in lower testing error than the anti-causal direction. However, existing regularization methods are agnostic of causality. We introduce Causal Structure Learning (CASTLE) regularization and propose to regularize a neural network by jointly learning the causal relationships between variables. CASTLE learns the causal directed acyclical graph (DAG) as an adjacency matrix embedded in the neural network's input layers, thereby facilitating the discovery of optimal predictors. Furthermore, CASTLE efficiently reconstructs only the features in the causal DAG that have a causal neighbor, whereas reconstruction-based regularizers suboptimally reconstruct all input features. We provide a theoretical generalization bound for our approach and conduct experiments on a plethora of synthetic and real publicly available datasets demonstrating that CASTLE consistently leads to better out-of-sample predictions as compared to other popular benchmark regularizers.


page 1

page 2

page 3

page 4


Bivariate Causal Discovery for Categorical Data via Classification with Optimal Label Permutation

Causal discovery for quantitative data has been extensively studied but ...

Causal Regularization Using Domain Priors

Neural networks leverage both causal and correlation-based relationships...

Physical System for Non Time Sequence Data

We propose a novelty approach to connect machine learning to causal stru...

Causal Modeling of Soil Processes for Improved Generalization

Measuring and monitoring soil organic carbon is critical for agricultura...

A probabilistic autoencoder for causal discovery

The paper addresses the problem of finding the causal direction between ...

Invariant Structure Learning for Better Generalization and Causal Explainability

Learning the causal structure behind data is invaluable for improving ge...

Causal Discovery by Kernel Intrinsic Invariance Measure

Reasoning based on causality, instead of association has been considered...