Fast Batch Nuclear-norm Maximization and Minimization for Robust Domain Adaptation

07/13/2021
by   Shuhao Cui, et al.
0

Due to the domain discrepancy in visual domain adaptation, the performance of source model degrades when bumping into the high data density near decision boundary in target domain. A common solution is to minimize the Shannon Entropy to push the decision boundary away from the high density area. However, entropy minimization also leads to severe reduction of prediction diversity, and unfortunately brings harm to the domain adaptation. In this paper, we investigate the prediction discriminability and diversity by studying the structure of the classification output matrix of a randomly selected data batch. We find by theoretical analysis that the prediction discriminability and diversity could be separately measured by the Frobenius-norm and rank of the batch output matrix. The nuclear-norm is an upperbound of the former, and a convex approximation of the latter. Accordingly, we propose Batch Nuclear-norm Maximization and Minimization, which performs nuclear-norm maximization on the target output matrix to enhance the target prediction ability, and nuclear-norm minimization on the source batch output matrix to increase applicability of the source domain knowledge. We further approximate the nuclear-norm by L_1,2-norm, and design multi-batch optimization for stable solution on large number of categories. The fast approximation method achieves O(n^2) computational complexity and better convergence property. Experiments show that our method could boost the adaptation accuracy and robustness under three typical domain adaptation scenarios. The code is available at https://github.com/cuishuhao/BNM.

READ FULL TEXT
research
03/27/2020

Towards Discriminability and Diversity: Batch Nuclear-norm Maximization under Label Insufficient Situations

The learning of the deep networks largely relies on the data with human-...
research
04/12/2022

On the Equity of Nuclear Norm Maximization in Unsupervised Domain Adaptation

Nuclear norm maximization has shown the power to enhance the transferabi...
research
02/02/2023

Confidence and Dispersity Speak: Characterising Prediction Matrix for Unsupervised Accuracy Estimation

This work aims to assess how well a model performs under distribution sh...
research
07/07/2021

Learning Invariant Representation with Consistency and Diversity for Semi-supervised Source Hypothesis Transfer

Semi-supervised domain adaptation (SSDA) aims to solve tasks in target d...
research
02/05/2020

Entropy Minimization vs. Diversity Maximization for Domain Adaptation

Entropy minimization has been widely used in unsupervised domain adaptat...
research
08/12/2022

Private Domain Adaptation from a Public Source

A key problem in a variety of applications is that of domain adaptation ...
research
02/05/2021

Co-Mixup: Saliency Guided Joint Mixup with Supermodular Diversity

While deep neural networks show great performance on fitting to the trai...

Please sign up or login with your details

Forgot password? Click here to reset