Matching Feature Sets for Few-Shot Image Classification

by   Arman Afrasiyabi, et al.

In image classification, it is common practice to train deep networks to extract a single feature vector per input image. Few-shot classification methods also mostly follow this trend. In this work, we depart from this established direction and instead propose to extract sets of feature vectors for each image. We argue that a set-based representation intrinsically builds a richer representation of images from the base classes, which can subsequently better transfer to the few-shot classes. To do so, we propose to adapt existing feature extractors to instead produce sets of feature vectors from images. Our approach, dubbed SetFeat, embeds shallow self-attention mechanisms inside existing encoder architectures. The attention modules are lightweight, and as such our method results in encoders that have approximately the same number of parameters as their original versions. During training and inference, a set-to-set matching metric is used to perform image classification. The effectiveness of our proposed architecture and metrics is demonstrated via thorough experiments on standard few-shot datasets – namely miniImageNet, tieredImageNet, and CUB – in both the 1- and 5-shot scenarios. In all cases but one, our method outperforms the state-of-the-art.


page 7

page 8

page 14


Task-Prior Conditional Variational Auto-Encoder for Few-Shot Image Classification

Transductive methods always outperform inductive methods in few-shot ima...

GCCN: Global Context Convolutional Network

In this paper, we propose Global Context Convolutional Network (GCCN) fo...

Deep Metric Learning for Few-Shot Image Classification: A Selective Review

Few-shot image classification is a challenging problem which aims to ach...

One-Shot Image Classification by Learning to Restore Prototypes

One-shot image classification aims to train image classifiers over the d...

Superpixel Image Classification with Graph Attention Networks

This document reports the use of Graph Attention Networks for classifyin...

Persistent Mixture Model Networks for Few-Shot Image Classification

We introduce Persistent Mixture Model (PMM) networks for representation ...

DeepEMD: Few-Shot Image Classification with Differentiable Earth Mover's Distance and Structured Classifiers

In this paper, we address the few-shot classification task from a new pe...

Please sign up or login with your details

Forgot password? Click here to reset