Multi-Level Contrastive Learning for Few-Shot Problems

07/15/2021
by   Qing Chen, et al.
0

Contrastive learning is a discriminative approach that aims at grouping similar samples closer and diverse samples far from each other. It it an efficient technique to train an encoder generating distinguishable and informative representations, and it may even increase the encoder's transferability. Most current applications of contrastive learning benefit only a single representation from the last layer of an encoder.In this paper, we propose a multi-level contrasitive learning approach which applies contrastive losses at different layers of an encoder to learn multiple representations from the encoder. Afterward, an ensemble can be constructed to take advantage of the multiple representations for the downstream tasks. We evaluated the proposed method on few-shot learning problems and conducted experiments using the mini-ImageNet and the tiered-ImageNet datasets. Our model achieved the new state-of-the-art results for both datasets, comparing to previous regular, ensemble, and contrastive learing (single-level) based approaches.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/25/2022

Multi-scale and Cross-scale Contrastive Learning for Semantic Segmentation

This work considers supervised contrastive learning for semantic segment...
research
01/20/2022

Watermarking Pre-trained Encoders in Contrastive Learning

Contrastive learning has become a popular technique to pre-train image e...
research
12/26/2020

Spatial Contrastive Learning for Few-Shot Classification

Existing few-shot classification methods rely to some degree on the cros...
research
10/14/2020

Function Contrastive Learning of Transferable Representations

Few-shot-learning seeks to find models that are capable of fast-adaptati...
research
05/04/2022

CoCa: Contrastive Captioners are Image-Text Foundation Models

Exploring large-scale pretrained foundation models is of significant int...
research
03/07/2023

A Light-Weight Contrastive Approach for Aligning Human Pose Sequences

We present a simple unsupervised method for learning an encoder mapping ...
research
01/15/2022

Multi-level Second-order Few-shot Learning

We propose a Multi-level Second-order (MlSo) few-shot learning network f...

Please sign up or login with your details

Forgot password? Click here to reset