FS-DETR: Few-Shot DEtection TRansformer with prompting and without re-training

10/10/2022
by   Adrian Bulat, et al.
14

This paper is on Few-Shot Object Detection (FSOD), where given a few templates (examples) depicting a novel class (not seen during training), the goal is to detect all of its occurrences within a set of images. From a practical perspective, an FSOD system must fulfil the following desiderata: (a) it must be used as is, without requiring any fine-tuning at test time, (b) it must be able to process an arbitrary number of novel objects concurrently while supporting an arbitrary number of examples from each class and (c) it must achieve accuracy comparable to a closed system. While there are (relatively) few systems that support (a), to our knowledge, there is no system supporting (b) and (c). In this work, we make the following contributions: We introduce, for the first time, a simple, yet powerful, few-shot detection transformer (FS-DETR) that can address both desiderata (a) and (b). Our system builds upon the DETR framework, extending it based on two key ideas: (1) feed the provided visual templates of the novel classes as visual prompts during test time, and (2) “stamp” these prompts with pseudo-class embeddings, which are then predicted at the output of the decoder. Importantly, we show that our system is not only more flexible than existing methods, but also, making a step towards satisfying desideratum (c), it is more accurate, matching and outperforming the current state-of-the-art on the most well-established benchmarks (PASCAL VOC MSCOCO) for FSOD. Code will be made available.

READ FULL TEXT

page 15

page 16

research
03/16/2020

Frustratingly Simple Few-Shot Object Detection

Detecting rare objects from a few examples is an emerging problem. Prior...
research
12/03/2020

Make One-Shot Video Object Segmentation Efficient Again

Video object segmentation (VOS) describes the task of segmenting a set o...
research
08/29/2022

CounTR: Transformer-based Generalised Visual Counting

In this paper, we consider the problem of generalised visual object coun...
research
10/26/2021

Plug-and-Play Few-shot Object Detection with Meta Strategy and Explicit Localization Inference

Aiming at recognizing and localizing the object of novel categories by a...
research
12/25/2018

Similarity R-C3D for Few-shot Temporal Activity Detection

Many activities of interest are rare events, with only a few labeled exa...
research
03/18/2021

GPT Understands, Too

While GPTs with traditional fine-tuning fail to achieve strong results o...
research
03/25/2022

Sylph: A Hypernetwork Framework for Incremental Few-shot Object Detection

We study the challenging incremental few-shot object detection (iFSD) se...

Please sign up or login with your details

Forgot password? Click here to reset