DeepAI AI Chat
Log In Sign Up

Conditional entropy minimization principle for learning domain invariant representation features

01/25/2022
by   Thuan Nguyen, et al.
Boston University
Tufts University
0

Invariance principle-based methods, for example, Invariant Risk Minimization (IRM), have recently emerged as promising approaches for Domain Generalization (DG). Despite the promising theory, invariance principle-based approaches fail in common classification tasks due to the mixture of the true invariant features and the spurious invariant features. In this paper, we propose a framework based on the conditional entropy minimization principle to filter out the spurious invariant features leading to a new algorithm with a better generalization capability. We theoretically prove that under some particular assumptions, the representation function can precisely recover the true invariant features. In addition, we also show that the proposed approach is closely related to the well-known Information Bottleneck framework. Both the theoretical and numerical results are provided to justify our approach.

READ FULL TEXT

page 1

page 2

page 3

page 4

06/11/2021

Invariance Principle Meets Information Bottleneck for Out-of-Distribution Generalization

The invariance principle from causality is at the heart of notable appro...
06/17/2021

On Invariance Penalties for Risk Minimization

The Invariant Risk Minimization (IRM) principle was first proposed by Ar...
08/16/2022

Counterfactual Supervision-based Information Bottleneck for Out-of-Distribution Generalization

Learning invariant (causal) features for out-of-distribution (OOD) gener...
01/28/2023

Learning Optimal Features via Partial Invariance

Learning models that are robust to test-time distribution shifts is a ke...
02/11/2022

Invariance Principle Meets Out-of-Distribution Generalization on Graphs

Despite recent developments in using the invariance principle from causa...
05/29/2022

The Missing Invariance Principle Found – the Reciprocal Twin of Invariant Risk Minimization

Machine learning models often generalize poorly to out-of-distribution (...
06/15/2022

Pareto Invariant Risk Minimization

Despite the success of invariant risk minimization (IRM) in tackling the...