Linear unit-tests for invariance discovery

07/24/2021
by   Agnieszka Słowik, et al.
0

There is an increasing interest in algorithms to learn invariant correlations across training environments. A big share of the current proposals find theoretical support in the causality literature but, how useful are they in practice? The purpose of this note is to propose six linear low-dimensional problems -- unit tests -- to evaluate different types of out-of-distribution generalization in a precise manner. Following initial experiments, none of the three recently proposed alternatives passes all tests. By providing the code to automatically replicate all the results in this manuscript (https://www.github.com/facebookresearch/InvarianceUnitTests), we hope that our unit tests become a standard steppingstone for researchers in out-of-distribution generalization.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/09/2017

A Unified Approach on the Local Power of Panel Unit Root Tests

In this paper, a unified approach is proposed to derive the exact local ...
research
04/12/2022

Toward Granular Automatic Unit Test Case Generation

Unit testing verifies the presence of faults in individual software comp...
research
04/30/2023

Exploring the Effectiveness of Large Language Models in Generating Unit Tests

A code generation model generates code by taking a prompt from a code co...
research
02/23/2020

Unit-root test within a threshold ARMA framework

We propose a new unit-root test based on Lagrange Multipliers, where we ...
research
12/19/2018

Carving Parameterized Unit Tests

We present a method to automatically extract ("carve") parameterized uni...
research
06/01/2023

In or Out? Fixing ImageNet Out-of-Distribution Detection Evaluation

Out-of-distribution (OOD) detection is the problem of identifying inputs...
research
08/06/2022

Class Is Invariant to Context and Vice Versa: On Learning Invariance for Out-Of-Distribution Generalization

Out-Of-Distribution generalization (OOD) is all about learning invarianc...

Please sign up or login with your details

Forgot password? Click here to reset