Unsupervised Embedding Learning from Uncertainty Momentum Modeling

07/19/2021
by   Jiahuan Zhou, et al.
20

Existing popular unsupervised embedding learning methods focus on enhancing the instance-level local discrimination of the given unlabeled images by exploring various negative data. However, the existed sample outliers which exhibit large intra-class divergences or small inter-class variations severely limit their learning performance. We justify that the performance limitation is caused by the gradient vanishing on these sample outliers. Moreover, the shortage of positive data and disregard for global discrimination consideration also pose critical issues for unsupervised learning but are always ignored by existing methods. To handle these issues, we propose a novel solution to explicitly model and directly explore the uncertainty of the given unlabeled learning samples. Instead of learning a deterministic feature point for each sample in the embedding space, we propose to represent a sample by a stochastic Gaussian with the mean vector depicting its space localization and covariance vector representing the sample uncertainty. We leverage such uncertainty modeling as momentum to the learning which is helpful to tackle the outliers. Furthermore, abundant positive candidates can be readily drawn from the learned instance-specific distributions which are further adopted to mitigate the aforementioned issues. Thorough rationale analyses and extensive experiments are presented to verify our superiority.

READ FULL TEXT

page 3

page 5

page 6

page 8

page 9

page 11

page 12

page 13

research
05/07/2021

Exploring Instance Relations for Unsupervised Feature Embedding

Despite the great progress achieved in unsupervised feature embedding, e...
research
09/09/2020

Deep Metric Learning Meets Deep Clustering: An Novel Unsupervised Approach for Feature Embedding

Unsupervised Deep Distance Metric Learning (UDML) aims to learn sample s...
research
09/30/2018

Modeling Uncertainty with Hedged Instance Embedding

Instance embeddings are an efficient and versatile image representation ...
research
03/21/2022

Unsupervised Heterophilous Network Embedding via r-Ego Network Discrimination

Recently, supervised network embedding (NE) has emerged as a predominant...
research
01/27/2022

Contrastive Embedding Distribution Refinement and Entropy-Aware Attention for 3D Point Cloud Classification

Learning a powerful representation from point clouds is a fundamental an...
research
05/15/2022

Real-centric Consistency Learning for Deepfake Detection

Most of previous deepfake detection researches bent their efforts to des...
research
09/01/2020

Graph Embedding with Data Uncertainty

spectral-based subspace learning is a common data preprocessing step in ...

Please sign up or login with your details

Forgot password? Click here to reset