Generalized Label Shift Correction via Minimum Uncertainty Principle: Theory and Algorithm

02/26/2022
by   You-Wei Luo, et al.
203

As a fundamental problem in machine learning, dataset shift induces a paradigm to learn and transfer knowledge under changing environment. Previous methods assume the changes are induced by covariate, which is less practical for complex real-world data. We consider the Generalized Label Shift (GLS), which provides an interpretable insight into the learning and transfer of desirable knowledge. Current GLS methods: 1) are not well-connected with the statistical learning theory; 2) usually assume the shifting conditional distributions will be matched with an implicit transformation, but its explicit modeling is unexplored. In this paper, we propose a conditional adaptation framework to deal with these challenges. From the perspective of learning theory, we prove that the generalization error of conditional adaptation is lower than previous covariate adaptation. Following the theoretical results, we propose the minimum uncertainty principle to learn conditional invariant transformation via discrepancy optimization. Specifically, we propose the conditional metric operator on Hilbert space to characterize the distinctness of conditional distributions. For finite observations, we prove that the empirical estimation is always well-defined and will converge to underlying truth as sample size increases. The results of extensive experiments demonstrate that the proposed model achieves competitive performance under different GLS scenarios.

READ FULL TEXT

page 1

page 6

page 11

page 13

07/31/2021

Conditional Bures Metric for Domain Adaptation

As a vital problem in classification-oriented transfer, unsupervised dom...
10/23/2019

Generalized Domain Adaptation with Covariate and Label Shift CO-ALignment

Unsupervised knowledge transfer has a great potential to improve the gen...
07/22/2016

On the Use of Sparse Filtering for Covariate Shift Adaptation

In this paper we formally analyse the use of sparse filtering algorithms...
03/23/2022

Towards Backwards-Compatible Data with Confounded Domain Adaptation

Most current domain adaptation methods address either covariate shift or...
11/15/2020

DIRL: Domain-Invariant Representation Learning for Sim-to-Real Transfer

Generating large-scale synthetic data in simulation is a feasible altern...
06/25/2021

Domain Conditional Predictors for Domain Adaptation

Learning guarantees often rely on assumptions of i.i.d. data, which will...
07/14/2022

Improved OOD Generalization via Conditional Invariant Regularizer

Recently, generalization on out-of-distribution (OOD) data with correlat...