On the Generalization and Adaption Performance of Causal Models

06/09/2022
by   Nino Scherrer, et al.
11

Learning models that offer robust out-of-distribution generalization and fast adaptation is a key challenge in modern machine learning. Modelling causal structure into neural networks holds the promise to accomplish robust zero and few-shot adaptation. Recent advances in differentiable causal discovery have proposed to factorize the data generating process into a set of modules, i.e. one module for the conditional distribution of every variable where only causal parents are used as predictors. Such a modular decomposition of knowledge enables adaptation to distributions shifts by only updating a subset of parameters. In this work, we systematically study the generalization and adaption performance of such modular neural causal models by comparing it to monolithic models and structured models where the set of predictors is not constrained to causal parents. Our analysis shows that the modular neural causal models outperform other models on both zero and few-shot adaptation in low data regimes and offer robust generalization. We also found that the effects are more significant for sparser graphs as compared to denser graphs.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
01/27/2023

Projected Subnetworks Scale Adaptation

Large models support great zero-shot and few-shot capabilities. However,...
research
06/14/2017

Schema Networks: Zero-shot Transfer with a Generative Causal Model of Intuitive Physics

The recent adaptation of deep neural network-based methods to reinforcem...
research
08/07/2022

Learning Modular Structures That Generalize Out-of-Distribution

Out-of-distribution (O.O.D.) generalization remains to be a key challeng...
research
07/18/2023

Batched Predictors Generalize within Distribution

We study the generalization properties of batched predictors, i.e., mode...
research
09/22/2021

Robust Generalization of Quadratic Neural Networks via Function Identification

A key challenge facing deep learning is that neural networks are often n...
research
04/16/2023

Out-of-Variable Generalization

The ability of an agent to perform well in new and unseen environments i...
research
07/13/2020

S2RMs: Spatially Structured Recurrent Modules

Capturing the structure of a data-generating process by means of appropr...

Please sign up or login with your details

Forgot password? Click here to reset