Active Self-Supervised Learning: A Few Low-Cost Relationships Are All You Need

03/27/2023
by   Vivien Cabannes, et al.
0

Self-Supervised Learning (SSL) has emerged as the solution of choice to learn transferable representations from unlabeled data. However, SSL requires to build samples that are known to be semantically akin, i.e. positive views. Requiring such knowledge is the main limitation of SSL and is often tackled by ad-hoc strategies e.g. applying known data-augmentations to the same input. In this work, we generalize and formalize this principle through Positive Active Learning (PAL) where an oracle queries semantic relationships between samples. PAL achieves three main objectives. First, it unveils a theoretically grounded learning framework beyond SSL, that can be extended to tackle supervised and semi-supervised learning depending on the employed oracle. Second, it provides a consistent algorithm to embed a priori knowledge, e.g. some observed labels, into any SSL losses without any change in the training pipeline. Third, it provides a proper active learning framework yielding low-cost solutions to annotate datasets, arguably bringing the gap between theory and practice of active learning that is based on simple-to-answer-by-non-experts queries of semantic relationships between inputs.

READ FULL TEXT

page 2

page 7

page 17

research
11/16/2020

On the Marginal Benefit of Active Learning: Does Self-Supervision Eat Its Cake?

Active learning is the set of techniques for intelligently labeling larg...
research
12/30/2022

Deep Active Learning Using Barlow Twins

The generalisation performance of a convolutional neural networks (CNN) ...
research
01/19/2022

Using Self-Supervised Pretext Tasks for Active Learning

Labeling a large set of data is expensive. Active learning aims to tackl...
research
01/22/2021

DSAL: Deeply Supervised Active Learning from Strong and Weak Labelers for Biomedical Image Segmentation

Image segmentation is one of the most essential biomedical image process...
research
02/08/2023

Revisiting Deep Active Learning for Semantic Segmentation

Active learning automatically selects samples for annotation from a data...
research
09/13/2022

Warm Start Active Learning with Proxy Labels & Selection via Semi-Supervised Fine-Tuning

Which volume to annotate next is a challenging problem in building medic...

Please sign up or login with your details

Forgot password? Click here to reset