TAX: Tendency-and-Assignment Explainer for Semantic Segmentation with Multi-Annotators

02/19/2023
by   Yuan-Chia Cheng, et al.
0

To understand how deep neural networks perform classification predictions, recent research attention has been focusing on developing techniques to offer desirable explanations. However, most existing methods cannot be easily applied for semantic segmentation; moreover, they are not designed to offer interpretability under the multi-annotator setting. Instead of viewing ground-truth pixel-level labels annotated by a single annotator with consistent labeling tendency, we aim at providing interpretable semantic segmentation and answer two critical yet practical questions: "who" contributes to the resulting segmentation, and "why" such an assignment is determined. In this paper, we present a learning framework of Tendency-and-Assignment Explainer (TAX), designed to offer interpretability at the annotator and assignment levels. More specifically, we learn convolution kernel subsets for modeling labeling tendencies of each type of annotation, while a prototype bank is jointly observed to offer visual guidance for learning the above kernels. For evaluation, we consider both synthetic and real-world datasets with multi-annotators. We show that our TAX can be applied to state-of-the-art network architectures with comparable performances, while segmentation interpretability at both levels can be offered accordingly.

READ FULL TEXT

page 3

page 6

page 7

research
02/28/2018

Learning to Adapt Structured Output Space for Semantic Segmentation

Convolutional neural network-based approaches for semantic segmentation ...
research
07/03/2020

Few-Shot Semantic Segmentation Augmented with Image-Level Weak Annotations

Despite the great progress made by deep neural networks in the semantic ...
research
07/30/2019

Grid Saliency for Context Explanations of Semantic Segmentation

Recently, there has been a growing interest in developing saliency metho...
research
04/13/2021

All you need are a few pixels: semantic segmentation with PixelPick

A central challenge for the task of semantic segmentation is the prohibi...
research
10/05/2021

MetaPix: Domain Transfer for Semantic Segmentation by Meta Pixel Weighting

Training a deep neural model for semantic segmentation requires collecti...
research
05/06/2019

Deep Visual City Recognition Visualization

Understanding how cities visually differ from each others is interesting...
research
12/29/2020

SALA: Soft Assignment Local Aggregation for 3D Semantic Segmentation

We introduce the idea of using learnable neighbor-to-grid soft assignmen...

Please sign up or login with your details

Forgot password? Click here to reset