Boosting Supervision with Self-Supervision for Few-shot Learning

06/17/2019
by   Jong-Chyi Su, et al.
6

We present a technique to improve the transferability of deep representations learned on small labeled datasets by introducing self-supervised tasks as auxiliary loss functions. While recent approaches for self-supervised learning have shown the benefits of training on large unlabeled datasets, we find improvements in generalization even on small datasets and when combined with strong supervision. Learning representations with self-supervised losses reduces the relative error rate of a state-of-the-art meta-learner by 5-25 several few-shot learning benchmarks, as well as off-the-shelf deep networks on standard classification tasks when training from scratch. We find the benefits of self-supervision increase with the difficulty of the task. Our approach utilizes the images within the dataset to construct self-supervised losses and hence is an effective way of learning transferable representations without relying on any external training data.

READ FULL TEXT

page 3

page 5

page 8

research
10/08/2019

When Does Self-supervision Improve Few-shot Learning?

We present a technique to improve the generalization of deep representat...
research
06/12/2019

Boosting Few-Shot Visual Learning with Self-Supervision

Few-shot learning and self-supervised learning address different facets ...
research
05/03/2022

Improving In-Context Few-Shot Learning via Self-Supervised Training

Self-supervised pretraining has made few-shot learning possible for many...
research
07/16/2021

Exploiting generative self-supervised learning for the assessment of biological images with lack of annotations: a COVID-19 case-study

Computer-aided analysis of biological images typically requires extensiv...
research
02/20/2023

Unsupervised Learning on a DIET: Datum IndEx as Target Free of Self-Supervision, Reconstruction, Projector Head

Costly, noisy, and over-specialized, labels are to be set aside in favor...
research
04/16/2021

Pareto Self-Supervised Training for Few-Shot Learning

While few-shot learning (FSL) aims for rapid generalization to new conce...
research
11/18/2022

Weighted Ensemble Self-Supervised Learning

Ensembling has proven to be a powerful technique for boosting model perf...

Please sign up or login with your details

Forgot password? Click here to reset