On the benefits of representation regularization in invariance based domain generalization

05/30/2021
by   Changjian Shui, et al.
0

A crucial aspect in reliable machine learning is to design a deployable system in generalizing new related but unobserved environments. Domain generalization aims to alleviate such a prediction gap between the observed and unseen environments. Previous approaches commonly incorporated learning invariant representation for achieving good empirical performance. In this paper, we reveal that merely learning invariant representation is vulnerable to the unseen environment. To this end, we derive novel theoretical analysis to control the unseen test environment error in the representation learning, which highlights the importance of controlling the smoothness of representation. In practice, our analysis further inspires an efficient regularization method to improve the robustness in domain generalization. Our regularization is orthogonal to and can be straightforwardly adopted in existing domain generalization algorithms for invariant representation learning. Empirical results show that our algorithm outperforms the base versions in various dataset and invariance criteria.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/29/2019

Adversarial Invariant Feature Learning with Accuracy Constraint for Domain Generalization

Learning domain-invariant representation is a dominant approach for doma...
research
04/25/2020

Convex Representation Learning for Generalized Invariance in Semi-Inner-Product Space

Invariance (defined in a general sense) has been one of the most effecti...
research
06/19/2023

Effect-Invariant Mechanisms for Policy Generalization

Policy learning is an important component of many real-world learning sy...
research
10/28/2022

Domain Generalization through the Lens of Angular Invariance

Domain generalization (DG) aims at generalizing a classifier trained on ...
research
03/02/2021

Generalizing to Unseen Domains: A Survey on Domain Generalization

Domain generalization (DG), i.e., out-of-distribution generalization, ha...
research
12/02/2019

Discovery and Separation of Features for Invariant Representation Learning

Supervised machine learning models often associate irrelevant nuisance f...
research
06/29/2022

When Does Group Invariant Learning Survive Spurious Correlations?

By inferring latent groups in the training data, recent works introduce ...

Please sign up or login with your details

Forgot password? Click here to reset