Imperfect Segmentation Labels: How Much Do They Matter?

06/12/2018
by   Nicholas Heller, et al.
0

Labeled datasets for semantic segmentation are imperfect, especially in medical imaging where borders are often subtle or ill-defined. Little work has been done to analyze the effect that label errors have on the performance of segmentation methodologies. Here we present a large-scale study of model performance in the presence of varying types and degrees of error in training data. We trained U-Net, SegNet, and FCN32 several times for liver segmentation with 10 different modes of ground-truth perturbation. Our results show that for each architecture, performance steadily declines with boundary-localized errors, however, U-Net was significantly more robust to jagged boundary errors than the other architectures. We also found that each architecture was very robust to non-boundary-localized errors, suggesting that boundary-localized errors are fundamentally different and more challenging problem than random label errors in a classification setting.

READ FULL TEXT

page 2

page 4

research
07/11/2023

Estimating label quality and errors in semantic segmentation data via any model

The labor-intensive annotation process of semantic segmentation datasets...
research
10/02/2020

Deep Learning for Earth Image Segmentation based on Imperfect Polyline Labels with Annotation Errors

In recent years, deep learning techniques (e.g., U-Net, DeepLab) have ac...
research
07/05/2021

Label noise in segmentation networks : mitigation must deal with bias

Imperfect labels limit the quality of predictions learned by deep neural...
research
12/30/2022

Informing selection of performance metrics for medical image segmentation evaluation using configurable synthetic errors

Machine learning-based segmentation in medical imaging is widely used in...
research
03/08/2015

TED: A Tolerant Edit Distance for Segmentation Evaluation

In this paper, we present a novel error measure to compare a segmentatio...
research
03/30/2021

Boundary IoU: Improving Object-Centric Image Segmentation Evaluation

We present Boundary IoU (Intersection-over-Union), a new segmentation ev...
research
12/01/2021

Investigation of Training Label Error Impact on RNN-T

In this paper, we propose an approach to quantitatively analyze impacts ...

Please sign up or login with your details

Forgot password? Click here to reset