Automatic segmentation of the pulmonary lobes with a 3D u-net and optimized loss function

by   Bianca Lassen-Schmidt, et al.

Fully-automatic lung lobe segmentation is challenging due to anatomical variations, pathologies, and incomplete fissures. We trained a 3D u-net for pulmonary lobe segmentation on 49 mainly publically available datasets and introduced a weighted Dice loss function to emphasize the lobar boundaries. To validate the performance of the proposed method we compared the results to two other methods. The new loss function improved the mean distance to 1.46 mm (compared to 2.08 mm for simple loss function without weighting).


A Novel Focal Tversky loss function with improved Attention U-Net for lesion segmentation

We propose a generalized focal loss function based on the Tversky index ...

Impact of loss function in Deep Learning methods for accurate retinal vessel segmentation

The retinal vessel network studied through fundus images contributes to ...

CT organ segmentation using GPU data augmentation, unsupervised labels and IOU loss

Fully-convolutional neural networks have achieved superior performance i...

A new method incorporating deep learning with shape priors for left ventricular segmentation in myocardial perfusion SPECT images

Background: The assessment of left ventricular (LV) function by myocardi...

Effective Training Strategies for Deep-learning-based Precipitation Nowcasting and Estimation

Deep learning has been successfully applied to precipitation nowcasting....

Interactive segmentation using U-Net with weight map and dynamic user interactions

Interactive segmentation has recently attracted attention for specialize...

iW-Net: an automatic and minimalistic interactive lung nodule segmentation deep network

We propose iW-Net, a deep learning model that allows for both automatic ...

1 Introduction

The human lungs are subdived into five lobes with separated supply branches for both vessels and airways. Usually, a visceral pleura called the pulmonary fissure can be found between adjacent lobes. Accurate segmentation of the five pulmonary lobes is important for diagnosis, treatment planning and monitoring for lung diseases such as COPD and fibrosis. In the last two decades several automatic approaches were developed [doel2015] each has different drawbacks and limitations. A few recently published approached are based on deep learning [gerard2019, wang2019, tang2019, park2019, lee2019, imran2018, george2017]. But none of them includes explicit knowledge from pulmonary fissures as weighting into the loss function. In this paper, we propose a fully automatic lobe segmentation method based on a 3D u-net with an optimized loss function focussing the lobar boundaries. We compare the new method with a proven fully automatic segmentation approach [lassen2013] that won the LOLA11 challenge [LOLA11] at publication time.

2 Method & Data

Pulmonary fissures are the most accurate feature for lobe segmentation but these are often incomplete and there can be accessory fissures in the lungs. We train a 3D u-net to rely on visible fissures and simultaneously learn that fissures are often incomplete.

Data: We use 70 lung CT scans, randomly subdivided into 49 training, 7 validation and 14 testing data sets. 50 datasets are taken from LIDC/IDRI [armato2011] with public available reference segmentations by Tang et al. [tang2019] and 20 datasets are from the University Medical Center Utrecht that are also used in [lassen2013]. Reference segmentations for the 20 datasets are generated with an automatic segmentation and interactive correction.


Figure 1: Training and segmentation process of the proposed method.

Training: We train two separate 3D u-nets (see Figure LABEL:fig:method), one for the left lung with 3 labels (background, upper lobe, lower lobe) and one for the right lung with 4 labels (background, upper lobe, middle lobe, lower lobe). We use 4 resolution levels, a filter size of

for the convolutions, PReLu activation, batch normalization and dropout.

The most promising features for lung lobe segmentation can be found at the lobar boundaries. We want our networks to learn two facts: 1. pulmonary fissures are at the lobar boundaries, but 2. there might be no fissures at the lobar boundaries (incomplete fissures). Therefore we introduce a weighted Dice loss in the following manner: We calculate an Euclidean distance transformation (EDT) from the lobar boundaries of the reference segmentation. Next, we invert the distance dist within a radius of 10 mm by setting the values to 10 - dist and set the remaining part of the lungs to a value of 1. This results in a weighting image with a value of 10 at the lobar boundaries descending to value 1 within 10 mm radius. We use the EDT because the reference segmentation might not be exact on the lobar boundaries and we do not want to miss information from visible fissures. Thus, we use the following loss function:

As a preprocessing step the input images are resampled to 1.5 . The 49 training datasets are subdivided into patches of size

voxels with 44 voxels padding on all six sides. We start the training with a learning rate of 0.005, a batchsize of 2 and stop after 70000 iterations.

Inference: For segmentation, we first resample a CT scan to 1.5 . Next, we apply our trained networks for the left and the right lung lobes and upsample the image back to the original resolution. We propose the following postprocessing to deal with remaining misclassifications regions (see Figure LABEL:fig:method): We perform a connected component analysis and keep the largest two (three) components in the left (right) lung. Finally, we fill the holes with a Voronoi division and use a given lung mask to delete all objects outside the lungs.

3 Evaluation & Results

We applied the described segmentation pipeline to the 14 testing datasets which were not used for training and validation. Segmentation including postprocessing takes less than 6 seconds for a case. We compared our method to two other approaches: 1. a non-deep-learning-based automatic method [lassen2013] 2. the same u-net as proposed but without weighting. The mean distance from the visible fissure improved to 1.46 mm (without weighting: 2.08 mm). See Figure 2 for plots and Figure 3 for screenshots.

Figure 2: Proposed method in comparison to a non-DL-based automatic method and a 3D u-net without weighting a) Dice coefficient b) mean distance to visible fissures
Figure 3: Segmentation results for two cases. a+e) CT scan, b+f) non-DL-based automatic method, c+g) 3D u-net without weighting, d+h) 3D u-net with weighting

4 Discussion

We trained a 3D u-net for a lung lobe segmentation task and showed that emphasizing the lobar boundaries in the loss function improved the segmentation results (see Figure 2 and 3). The segmentation quality is comparable to the method proposed in [lassen2013] and even slightly better for the left lobes. This study was performed on a small amount of data. In future work, we plan to train with the same architecture on a much larger database including a wide range of pathologies and performing an extensive evaluation with participation in the LOLA11 [LOLA11] challenge.