
On Asymptotic Standard Normality of the Two Sample Pivot
The asymptotic solution to the problem of comparing the means of two het...
read it

Multireference alignment in high dimensions: sample complexity and phase transition
Multireference alignment entails estimating a signal in ℝ^L from its ci...
read it

Consistency and Finite Sample Behavior of Binary Class Probability Estimation
In this work we investigate to which extent one can recover class probab...
read it

Minimax Learning of Ergodic Markov Chains
We compute the finitesample minimax (modulo logarithmic factors) sample...
read it

The Risks of Invariant Risk Minimization
Invariant Causal Prediction (Peters et al., 2016) is a technique for out...
read it

Linear Regression Games: Convergence Guarantees to Approximate OutofDistribution Solutions
Recently, invariant risk minimization (IRM) (Arjovsky et al.) was propos...
read it

Statistical Analysis of Stationary Solutions of Coupled Nonconvex Nonsmooth Empirical Risk Minimization
This paper has two main goals: (a) establish several statistical propert...
read it
Empirical or Invariant Risk Minimization? A Sample Complexity Perspective
Recently, invariant risk minimization (IRM) was proposed as a promising solution to address outofdistribution (OOD) generalization. However, it is unclear when IRM should be preferred over the widelyemployed empirical risk minimization (ERM) framework. In this work, we analyze both these frameworks from the perspective of sample complexity, thus taking a firm step towards answering this important question. We find that depending on the type of data generation mechanism, the two approaches might have very different finite sample and asymptotic behavior. For example, in the covariate shift setting we see that the two approaches not only arrive at the same asymptotic solution, but also have similar finite sample behavior with no clear winner. For other distribution shifts such as those involving confounders or anticausal variables, however, the two approaches arrive at different asymptotic solutions where IRM is guaranteed to be close to the desired OOD solutions in the finite sample regime, while ERM is biased even asymptotically. We further investigate how different factors – the number of environments, complexity of the model, and IRM penalty weight – impact the sample complexity of IRM in relation to its distance from the OOD solutions
READ FULL TEXT
Comments
There are no comments yet.