DeepAI
Log In Sign Up

Prototype Mixture Models for Few-shot Semantic Segmentation

08/10/2020
by   Boyu Yang, et al.
9

Few-shot segmentation is challenging because objects within the support and query images could significantly differ in appearance and pose. Using a single prototype acquired directly from the support image to segment the query image causes semantic ambiguity. In this paper, we propose prototype mixture models (PMMs), which correlate diverse image regions with multiple prototypes to enforce the prototype-based semantic representation. Estimated by an Expectation-Maximization algorithm, PMMs incorporate rich channel-wised and spatial semantics from limited support images. Utilized as representations as well as classifiers, PMMs fully leverage the semantics to activate objects in the query image while depressing background regions in a duplex manner. Extensive experiments on Pascal VOC and MS-COCO datasets show that PMMs significantly improve upon state-of-the-arts. Particularly, PMMs improve 5-shot segmentation performance on MS-COCO by up to 5.82% with only a moderate cost for model size and inference speed.

READ FULL TEXT

page 10

page 11

04/19/2021

SCNet: Enhancing Few-Shot Semantic Segmentation by Self-Contrastive Background Prototypes

Few-shot semantic segmentation aims to segment novel-class objects in a ...
03/17/2022

Multi-similarity based Hyperrelation Network for few-shot segmentation

Few-shot semantic segmentation aims at recognizing the object regions of...
02/10/2020

Prototype Refinement Network for Few-Shot Segmentation

Few-shot segmentation targets to segment new classes with few annotated ...
05/14/2021

Attentional Prototype Inference for Few-Shot Semantic Segmentation

This paper aims to address few-shot semantic segmentation. While existin...
11/23/2020

BiOpt: Bi-Level Optimization for Few-Shot Segmentation

Few-shot segmentation is a challenging task that aims to segment objects...
02/22/2021

Semantically Meaningful Class Prototype Learning for One-Shot Image Semantic Segmentation

One-shot semantic image segmentation aims to segment the object regions ...
10/03/2022

Few-Shot Segmentation via Rich Prototype Generation and Recurrent Prediction Enhancement

Prototype learning and decoder construction are the keys for few-shot se...