DeepAI AI Chat
Log In Sign Up

Kernelized Heterogeneous Risk Minimization

10/24/2021
by   Jiashuo Liu, et al.
Tsinghua University
0

The ability to generalize under distributional shifts is essential to reliable machine learning, while models optimized with empirical risk minimization usually fail on non-i.i.d testing data. Recently, invariant learning methods for out-of-distribution (OOD) generalization propose to find causally invariant relationships with multi-environments. However, modern datasets are frequently multi-sourced without explicit source labels, rendering many invariant learning methods inapplicable. In this paper, we propose Kernelized Heterogeneous Risk Minimization (KerHRM) algorithm, which achieves both the latent heterogeneity exploration and invariant learning in kernel space, and then gives feedback to the original neural network by appointing invariant gradient direction. We theoretically justify our algorithm and empirically validate the effectiveness of our algorithm with extensive experiments.

READ FULL TEXT

page 1

page 2

page 3

page 4

05/09/2021

Heterogeneous Risk Minimization

Machine learning algorithms with empirical risk minimization usually suf...
05/22/2023

Conformal Inference for Invariant Risk Minimization

The application of machine learning models can be significantly impeded ...
06/15/2022

Pareto Invariant Risk Minimization

Despite the success of invariant risk minimization (IRM) in tackling the...
06/13/2020

Risk Variance Penalization: From Distributional Robustness to Causality

Learning under multi-environments often requires the ability of out-of-d...
06/29/2022

When Does Group Invariant Learning Survive Spurious Correlations?

By inferring latent groups in the training data, recent works introduce ...
11/15/2022

Empirical Study on Optimizer Selection for Out-of-Distribution Generalization

Modern deep learning systems are fragile and do not generalize well unde...
06/08/2020

Invariant Adversarial Learning for Distributional Robustness

Machine learning algorithms with empirical risk minimization are vulnera...