DeepAI AI Chat
Log In Sign Up

The Difficult Task of Distribution Generalization in Nonlinear Models

by   Rune Christiansen, et al.

We consider the problem of predicting a response from a set of covariates when the test distribution differs from the training distribution. Here, we consider robustness against distributions that emerge as intervention distributions. Causal models that regress the response variable on all of its causal parents have been suggested for the above task since they remain valid under arbitrary interventions on any subset of covariates. However, in linear models, for a set of interventions with bounded strength, alternative approaches have been shown to be minimax prediction optimal. In this work, we analyze minimax solutions in nonlinear models for both direct and indirect interventions on the covariates. We prove that the causal function is minimax optimal for a large class of interventions. We introduce the notion of distribution generalization, which is motivated by the fact that, in practice, minimax solutions need to be identified from observational data. We prove sufficient conditions for distribution generalization and present corresponding impossibility results. To illustrate the above findings, we propose a practical method, called NILE, that achieves distribution generalization in a nonlinear instrumental variable setting with linear extrapolation. We prove consistency, present empirical results and provide code.


page 1

page 2

page 3

page 4


Regularizing towards Causal Invariance: Linear Models with Proxies

We propose a method for learning linear models whose predictive performa...

Nonparametric Identifiability of Causal Representations from Unknown Interventions

We study causal representation learning, the task of inferring latent ca...

Learning Optimal Interventions

Our goal is to identify beneficial interventions from observational data...

Active Invariant Causal Prediction: Experiment Selection through Stability

A fundamental difficulty of causal learning is that causal models can ge...

PAC Generalization via Invariant Representations

One method for obtaining generalizable solutions to machine learning tas...

Causality and Generalizability: Identifiability and Learning Methods

This PhD thesis contains several contributions to the field of statistic...

First Steps Toward Understanding the Extrapolation of Nonlinear Models to Unseen Domains

Real-world machine learning applications often involve deploying neural ...

Code Repositories