DeepAI AI Chat
Log In Sign Up

Conditional entropy minimization principle for learning domain invariant representation features

by   Thuan Nguyen, et al.
Boston University
Tufts University

Invariance principle-based methods, for example, Invariant Risk Minimization (IRM), have recently emerged as promising approaches for Domain Generalization (DG). Despite the promising theory, invariance principle-based approaches fail in common classification tasks due to the mixture of the true invariant features and the spurious invariant features. In this paper, we propose a framework based on the conditional entropy minimization principle to filter out the spurious invariant features leading to a new algorithm with a better generalization capability. We theoretically prove that under some particular assumptions, the representation function can precisely recover the true invariant features. In addition, we also show that the proposed approach is closely related to the well-known Information Bottleneck framework. Both the theoretical and numerical results are provided to justify our approach.


page 1

page 2

page 3

page 4


Invariance Principle Meets Information Bottleneck for Out-of-Distribution Generalization

The invariance principle from causality is at the heart of notable appro...

On Invariance Penalties for Risk Minimization

The Invariant Risk Minimization (IRM) principle was first proposed by Ar...

Counterfactual Supervision-based Information Bottleneck for Out-of-Distribution Generalization

Learning invariant (causal) features for out-of-distribution (OOD) gener...

Learning Optimal Features via Partial Invariance

Learning models that are robust to test-time distribution shifts is a ke...

Invariance Principle Meets Out-of-Distribution Generalization on Graphs

Despite recent developments in using the invariance principle from causa...

The Missing Invariance Principle Found – the Reciprocal Twin of Invariant Risk Minimization

Machine learning models often generalize poorly to out-of-distribution (...

Pareto Invariant Risk Minimization

Despite the success of invariant risk minimization (IRM) in tackling the...