DeepAI AI Chat
Log In Sign Up

Pareto Invariant Risk Minimization

by   Yongqiang Chen, et al.
The Chinese University of Hong Kong

Despite the success of invariant risk minimization (IRM) in tackling the Out-of-Distribution generalization problem, IRM can compromise the optimality when applied in practice. The practical variants of IRM, e.g., IRMv1, have been shown to have significant gaps with IRM and thus could fail to capture the invariance even in simple problems. Moreover, the optimization procedure in IRMv1 involves two intrinsically conflicting objectives, and often requires careful tuning for the objective weights. To remedy the above issues, we reformulate IRM as a multi-objective optimization problem, and propose a new optimization scheme for IRM, called PAreto Invariant Risk Minimization (PAIR). PAIR can adaptively adjust the optimization direction under the objective conflicts. Furthermore, we show PAIR can empower the practical IRM variants to overcome the barriers with the original IRM when provided with proper guidance. We conduct experiments with ColoredMNIST to confirm our theory and the effectiveness of PAIR.


page 2

page 3


Does Invariant Risk Minimization Capture Invariance?

We show that the Invariant Risk Minimization (IRM) formulation of Arjovs...

Invariant Risk Minimization

We introduce Invariant Risk Minimization (IRM), a learning paradigm to e...

Kernelized Heterogeneous Risk Minimization

The ability to generalize under distributional shifts is essential to re...

What Is Missing in IRM Training and Evaluation? Challenges and Solutions

Invariant risk minimization (IRM) has received increasing attention as a...

Learning Optimal Features via Partial Invariance

Learning models that are robust to test-time distribution shifts is a ke...

Conditional entropy minimization principle for learning domain invariant representation features

Invariance principle-based methods, for example, Invariant Risk Minimiza...

A Review of the Deep Sea Treasure problem as a Multi-Objective Reinforcement Learning Benchmark

In this paper, the authors investigate the Deep Sea Treasure (DST) probl...