DeepAI
Log In Sign Up

Linear Regression Games: Convergence Guarantees to Approximate Out-of-Distribution Solutions

10/28/2020
by   Kartik Ahuja, et al.
0

Recently, invariant risk minimization (IRM) (Arjovsky et al.) was proposed as a promising solution to address out-of-distribution (OOD) generalization. In Ahuja et al., it was shown that solving for the Nash equilibria of a new class of "ensemble-games" is equivalent to solving IRM. In this work, we extend the framework in Ahuja et al. for linear regressions by projecting the ensemble-game on an ℓ_∞ ball. We show that such projections help achieve non-trivial OOD guarantees despite not achieving perfect invariance. For linear models with confounders, we prove that Nash equilibria of these games are closer to the ideal OOD solutions than the standard empirical risk minimization (ERM) and we also provide learning algorithms that provably converge to these Nash Equilibria. Empirical comparisons of the proposed approach with the state-of-the-art show consistent gains in achieving OOD solutions in several settings involving anti-causal variables and confounders.

READ FULL TEXT

page 1

page 2

page 3

page 4

02/11/2020

Invariant Risk Minimization Games

The standard risk minimization paradigm of machine learning is brittle w...
08/16/2022

Counterfactual Supervision-based Information Bottleneck for Out-of-Distribution Generalization

Learning invariant (causal) features for out-of-distribution (OOD) gener...
12/04/2019

On games with coordinating and anti-coordinating agents

This work studies Nash equilibria for heterogeneous games where both coo...
11/08/2021

Best-Response Dynamics and Fictitious Play in Identical Interest Stochastic Games

This paper combines ideas from Q-learning and fictitious play to define ...
06/03/2021

Tropical linear regression and mean payoff games: or, how to measure the distance to equilibria

We study a tropical linear regression problem consisting in finding the ...
10/12/2020

The Risks of Invariant Risk Minimization

Invariant Causal Prediction (Peters et al., 2016) is a technique for out...
11/24/2018

On Independent Cliques and Linear Complementarity Problems

In recent work (Pandit and Kulkarni [Discrete Applied Mathematics, 244 (...