DeepAI AI Chat
Log In Sign Up

DUEL: Adaptive Duplicate Elimination on Working Memory for Self-Supervised Learning

10/31/2022
by   Won-Seok Choi, et al.
Seoul National University
1

In Self-Supervised Learning (SSL), it is known that frequent occurrences of the collision in which target data and its negative samples share the same class can decrease performance. Especially in real-world data such as crawled data or robot-gathered observations, collisions may occur more often due to the duplicates in the data. To deal with this problem, we claim that sampling negative samples from the adaptively debiased distribution in the memory makes the model more stable than sampling from a biased dataset directly. In this paper, we introduce a novel SSL framework with adaptive Duplicate Elimination (DUEL) inspired by the human working memory. The proposed framework successfully prevents the downstream task performance from degradation due to a dramatic inter-class imbalance.

READ FULL TEXT
11/26/2020

Evaluation of Out-of-Distribution Detection Performance of Self-Supervised Learning in a Controllable Environment

We evaluate the out-of-distribution (OOD) detection performance of self-...
10/11/2021

Self-supervised Learning is More Robust to Dataset Imbalance

Self-supervised learning (SSL) is a scalable way to learn general visual...
12/15/2021

Improving Self-supervised Learning with Automated Unsupervised Outlier Arbitration

Our work reveals a structured shortcoming of the existing mainstream sel...
09/07/2021

Self-supervised Tumor Segmentation through Layer Decomposition

In this paper, we propose a self-supervised approach for tumor segmentat...
11/22/2020

Run Away From your Teacher: Understanding BYOL by a Novel Self-Supervised Approach

Recently, a newly proposed self-supervised framework Bootstrap Your Own ...
12/31/2020

A Constant-time Adaptive Negative Sampling

Softmax classifiers with a very large number of classes naturally occur ...