Rethinking Semantic Segmentation Evaluation for Explainability and Model Selection

by   Yuxiang Zhang, et al.

Semantic segmentation aims to robustly predict coherent class labels for entire regions of an image. It is a scene understanding task that powers real-world applications (e.g., autonomous navigation). One important application, the use of imagery for automated semantic understanding of pedestrian environments, provides remote mapping of accessibility features in street environments. This application (and others like it) require detailed geometric information of geographical objects. Semantic segmentation is a prerequisite for this task since it maps contiguous regions of the same class as single entities. Importantly, semantic segmentation uses like ours are not pixel-wise outcomes; however, most of their quantitative evaluation metrics (e.g., mean Intersection Over Union) are based on pixel-wise similarities to a ground-truth, which fails to emphasize over- and under-segmentation properties of a segmentation model. Here, we introduce a new metric to assess region-based over- and under-segmentation. We analyze and compare it to other metrics, demonstrating that the use of our metric lends greater explainability to semantic segmentation model performance in real-world applications.


page 1

page 2

page 3

page 8

page 12

page 13

page 14


Weighted Intersection over Union (wIoU): A New Evaluation Metric for Image Segmentation

In this paper, we propose a novel evaluation metric for performance eval...

Diagnostics in Semantic Segmentation

Over the past years, computer vision community has contributed to enormo...

Learning to segment from misaligned and partial labels

To extract information at scale, researchers increasingly apply semantic...

Sensor Adaptation for Improved Semantic Segmentation of Overhead Imagery

Semantic segmentation is a powerful method to facilitate visual scene un...

Amodal Panoptic Segmentation

Humans have the remarkable ability to perceive objects as a whole, even ...

Global and Local Features through Gaussian Mixture Models on Image Semantic Segmentation

The semantic segmentation task aims at dense classification at the pixel...