Log In Sign Up

ZIN: When and How to Learn Invariance by Environment Inference?

by   Yong Lin, et al.

It is commonplace to encounter heterogeneous data, of which some aspects of the data distribution may vary but the underlying causal mechanisms remain constant. When data are divided into distinct environments according to the heterogeneity, recent invariant learning methods have proposed to learn robust and invariant models based on this environment partition. It is hence tempting to utilize the inherent heterogeneity even when environment partition is not provided. Unfortunately, in this work, we show that learning invariant features under this circumstance is fundamentally impossible without further inductive biases or additional information. Then, we propose a framework to jointly learn environment partition and invariant representation, assisted by additional auxiliary information. We derive sufficient and necessary conditions for our framework to provably identify invariant features under a fairly general setting. Experimental results on both synthetic and real world datasets validate our analysis and demonstrate an improved performance of the proposed framework over existing methods. Finally, our results also raise the need of making the role of inductive biases more explicit in future works, when considering learning invariant models without environment partition.


page 1

page 2

page 3

page 4


Repeated Environment Inference for Invariant Learning

We study the problem of invariant learning when the environment labels a...

A Survey of Inductive Biases for Factorial Representation-Learning

With the resurgence of interest in neural networks, representation learn...

Differentiable Invariant Causal Discovery

Learning causal structure from observational data is a fundamental chall...

Distributionally Invariant Learning: Rationalization and Practical Algorithms

The invariance property across environments is at the heart of invariant...

The Risks of Invariant Risk Minimization

Invariant Causal Prediction (Peters et al., 2016) is a technique for out...

Identifying Invariant Factors Across Multiple Environments with KL Regression

Many datasets are collected from multiple environments (e.g. different l...

On the inductive biases of deep domain adaptation

Domain alignment is currently the most prevalent solution to unsupervise...