DeepAI
Log In Sign Up

Out-of-Distribution Generalization with Maximal Invariant Predictor

08/04/2020
by   Masanori Koyama, et al.
0

Out-of-Distribution (OOD) generalization problem is a problem of seeking the predictor function whose performance in the worst environments is optimal. This paper makes two contributions to OOD problem. We first use the basic results of probability to prove maximal Invariant Predictor(MIP) condition, a theoretical result that can be used to identify the OOD optimal solution. We then use our MIP to derive inner-environmental Gradient Alignment(IGA) algorithm that can be used to help seek the OOD optimal predictor. Previous studies that have investigated the theoretical aspect of the OOD-problem use strong structural assumptions such as causal DAG. However, in cases involving image datasets, for example, the identification of hidden structural relations is itself a difficult problem. Our theoretical results are different from those of many previous studies in that it can be applied to cases in which the underlying structure of a dataset is difficult to analyze. We present an extensive comparison of previous theoretical approaches to the OODproblems based on the assumptions they make. We also present an extension of the colored-MNIST that can more accurately represent the pathological OOD situation than the original version, and demonstrate the superiority of IGA over previous methods on both the original and the extended version of Colored-MNIST.

READ FULL TEXT
05/04/2021

Robust Reconfigurable Intelligent Surfaces via Invariant Risk and Causal Representations

In this paper, the problem of robust reconfigurable intelligent surface ...
07/05/2021

Causally Invariant Predictor with Shift-Robustness

This paper proposes an invariant causal predictor that is robust to dist...
11/17/2019

Learning The MMSE Channel Predictor

We present a neural network based predictor which is derived by starting...
10/27/2021

Learning-Augmented k-means Clustering

k-means clustering is a well-studied problem due to its wide applicabili...
10/12/2020

The Risks of Invariant Risk Minimization

Invariant Causal Prediction (Peters et al., 2016) is a technique for out...
07/12/2019

On a Generalization of the Marriage Problem

We present a generalization of the marriage problem underlying Hall's fa...
02/11/2022

Distributionally Robust Data Join

Suppose we are given two datasets: a labeled dataset and unlabeled datas...