Refining Self-Supervised Learning in Imaging: Beyond Linear Metric

02/25/2022
by   Bo Jiang, et al.
13

We introduce in this paper a new statistical perspective, exploiting the Jaccard similarity metric, as a measure-based metric to effectively invoke non-linear features in the loss of self-supervised contrastive learning. Specifically, our proposed metric may be interpreted as a dependence measure between two adapted projections learned from the so-called latent representations. This is in contrast to the cosine similarity measure in the conventional contrastive learning model, which accounts for correlation information. To the best of our knowledge, this effectively non-linearly fused information embedded in the Jaccard similarity, is novel to self-supervision learning with promising results. The proposed approach is compared to two state-of-the-art self-supervised contrastive learning methods on three image datasets. We not only demonstrate its amenable applicability in current ML problems, but also its improved performance and training efficiency.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/15/2022

SCD: Self-Contrastive Decorrelation for Sentence Embeddings

In this paper, we propose Self-Contrastive Decorrelation (SCD), a self-s...
research
01/08/2023

Learning the Relation between Similarity Loss and Clustering Loss in Self-Supervised Learning

Self-supervised learning enables networks to learn discriminative featur...
research
03/05/2021

Extending Contrastive Learning to Unsupervised Coreset Selection

Self-supervised contrastive learning offers a means of learning informat...
research
01/12/2021

Explicit homography estimation improves contrastive self-supervised learning

The typical contrastive self-supervised algorithm uses a similarity meas...
research
10/17/2022

Self-Supervised Learning Through Efference Copies

Self-supervised learning (SSL) methods aim to exploit the abundance of u...
research
08/24/2022

SCALE: Online Self-Supervised Lifelong Learning without Prior Knowledge

Unsupervised lifelong learning refers to the ability to learn over time ...
research
09/02/2021

Self-supervised Recommendation with Cross-channel Matching Representation and Hierarchical Contrastive Learning

Recently, using different channels to model social semantic information,...

Please sign up or login with your details

Forgot password? Click here to reset