Multi-task Fusion for Efficient Panoptic-Part Segmentation

12/15/2022
by   Sravan Kumar Jagadeesh, et al.
0

In this paper, we introduce a novel network that generates semantic, instance, and part segmentation using a shared encoder and effectively fuses them to achieve panoptic-part segmentation. Unifying these three segmentation problems allows for mutually improved and consistent representation learning. To fuse the predictions of all three heads efficiently, we introduce a parameter-free joint fusion module that dynamically balances the logits and fuses them to create panoptic-part segmentation. Our method is evaluated on the Cityscapes Panoptic Parts (CPP) and Pascal Panoptic Parts (PPP) datasets. For CPP, the PartPQ of our proposed model with joint fusion surpasses the previous state-of-the-art by 1.6 and 4.7 percentage points for all areas and segments with parts, respectively. On PPP, our joint fusion outperforms a model using the previous top-down merging strategy by 3.3 percentage points in PartPQ and 10.5 percentage points in PartPQ for partitionable classes.

READ FULL TEXT

page 2

page 5

page 8

page 11

page 12

research
11/16/2021

Robust 3D Scene Segmentation through Hierarchical and Learnable Part-Fusion

3D semantic segmentation is a fundamental building block for several sce...
research
11/19/2020

Unifying Instance and Panoptic Segmentation with Dynamic Rank-1 Convolutions

Recently, fully-convolutional one-stage networks have shown superior per...
research
01/21/2020

Joint Learning of Instance and Semantic Segmentation for Robotic Pick-and-Place with Heavy Occlusions in Clutter

We present joint learning of instance and semantic segmentation for visi...
research
03/30/2016

Confidence driven TGV fusion

We introduce a novel model for spatially varying variational data fusion...
research
05/01/2015

Joint Object and Part Segmentation using Deep Learned Potentials

Segmenting semantic objects from images and parsing them into their resp...
research
11/29/2020

ProtoPShare: Prototype Sharing for Interpretable Image Classification and Similarity Discovery

In this paper, we introduce ProtoPShare, a self-explained method that in...
research
09/07/2019

Road Mapping In LiDAR Images Using A Joint-Task Dense Dilated Convolutions Merging Network

It is important, but challenging, for the forest industry to accurately ...

Please sign up or login with your details

Forgot password? Click here to reset