Generalized Label Shift Correction via Minimum Uncertainty Principle: Theory and Algorithm

02/26/2022
by   You-Wei Luo, et al.
203

As a fundamental problem in machine learning, dataset shift induces a paradigm to learn and transfer knowledge under changing environment. Previous methods assume the changes are induced by covariate, which is less practical for complex real-world data. We consider the Generalized Label Shift (GLS), which provides an interpretable insight into the learning and transfer of desirable knowledge. Current GLS methods: 1) are not well-connected with the statistical learning theory; 2) usually assume the shifting conditional distributions will be matched with an implicit transformation, but its explicit modeling is unexplored. In this paper, we propose a conditional adaptation framework to deal with these challenges. From the perspective of learning theory, we prove that the generalization error of conditional adaptation is lower than previous covariate adaptation. Following the theoretical results, we propose the minimum uncertainty principle to learn conditional invariant transformation via discrepancy optimization. Specifically, we propose the conditional metric operator on Hilbert space to characterize the distinctness of conditional distributions. For finite observations, we prove that the empirical estimation is always well-defined and will converge to underlying truth as sample size increases. The results of extensive experiments demonstrate that the proposed model achieves competitive performance under different GLS scenarios.

READ FULL TEXT

page 1

page 6

page 11

page 13

research
07/31/2021

Conditional Bures Metric for Domain Adaptation

As a vital problem in classification-oriented transfer, unsupervised dom...
research
10/23/2019

Generalized Domain Adaptation with Covariate and Label Shift CO-ALignment

Unsupervised knowledge transfer has a great potential to improve the gen...
research
09/17/2022

Mitigating Both Covariate and Conditional Shift for Domain Generalization

Domain generalization (DG) aims to learn a model on several source domai...
research
07/22/2016

On the Use of Sparse Filtering for Covariate Shift Adaptation

In this paper we formally analyse the use of sparse filtering algorithms...
research
03/23/2022

Towards Backwards-Compatible Data with Confounded Domain Adaptation

Most current domain adaptation methods address either covariate shift or...
research
12/02/2022

Stable Learning via Sparse Variable Independence

The problem of covariate-shift generalization has attracted intensive re...
research
11/15/2020

DIRL: Domain-Invariant Representation Learning for Sim-to-Real Transfer

Generating large-scale synthetic data in simulation is a feasible altern...

Please sign up or login with your details

Forgot password? Click here to reset