DeepAI AI Chat
Log In Sign Up

Reference Twice: A Simple and Unified Baseline for Few-Shot Instance Segmentation

by   Yue Han, et al.
Zhejiang University
Peking University

Few Shot Instance Segmentation (FSIS) requires models to detect and segment novel classes with limited several support examples. In this work, we explore a simple yet unified solution for FSIS as well as its incremental variants, and introduce a new framework named Reference Twice (RefT) to fully explore the relationship between support/query features based on a Transformer-like framework. Our key insights are two folds: Firstly, with the aid of support masks, we can generate dynamic class centers more appropriately to re-weight query features. Secondly, we find that support object queries have already encoded key factors after base training. In this way, the query features can be enhanced twice from two aspects, i.e., feature-level and instance-level. In particular, we firstly design a mask-based dynamic weighting module to enhance support features and then propose to link object queries for better calibration via cross-attention. After the above steps, the novel classes can be improved significantly over our strong baseline. Additionally, our new framework can be easily extended to incremental FSIS with minor modification. When benchmarking results on the COCO dataset for FSIS, gFSIS, and iFSIS settings, our method achieves a competitive performance compared to existing approaches across different shots, e.g., we boost nAP by noticeable +8.2/+9.4 over the current state-of-the-art FSIS method for 10/30-shot. We further demonstrate the superiority of our approach on Few Shot Object Detection. Code and model will be available.


page 3

page 4

page 8


SOLQ: Segmenting Objects by Learning Queries

In this paper, we propose an end-to-end framework for instance segmentat...

Incremental Few-Shot Instance Segmentation

Few-shot instance segmentation methods are promising when labeled traini...

One-Shot Instance Segmentation

We tackle one-shot visual search by example for arbitrary object categor...

FGN: Fully Guided Network for Few-Shot Instance Segmentation

Few-shot instance segmentation (FSIS) conjoins the few-shot learning par...

Semantic-aligned Fusion Transformer for One-shot Object Detection

One-shot object detection aims at detecting novel objects according to m...

PanopticPartFormer++: A Unified and Decoupled View for Panoptic Part Segmentation

Panoptic Part Segmentation (PPS) unifies panoptic segmentation and part ...

Betrayed by Captions: Joint Caption Grounding and Generation for Open Vocabulary Instance Segmentation

In this work, we focus on instance-level open vocabulary segmentation, i...