Edge-competing Pathological Liver Vessel Segmentation with Limited Labels

08/01/2021 ∙ by Zunlei Feng, et al. ∙ Zhejiang University Stevens Institute of Technology 0

The microvascular invasion (MVI) is a major prognostic factor in hepatocellular carcinoma, which is one of the malignant tumors with the highest mortality rate. The diagnosis of MVI needs discovering the vessels that contain hepatocellular carcinoma cells and counting their number in each vessel, which depends heavily on experiences of the doctor, is largely subjective and time-consuming. However, there is no algorithm as yet tailored for the MVI detection from pathological images. This paper collects the first pathological liver image dataset containing 522 whole slide images with labels of vessels, MVI, and hepatocellular carcinoma grades. The first and essential step for the automatic diagnosis of MVI is the accurate segmentation of vessels. The unique characteristics of pathological liver images, such as super-large size, multi-scale vessel, and blurred vessel edges, make the accurate vessel segmentation challenging. Based on the collected dataset, we propose an Edge-competing Vessel Segmentation Network (EVS-Net), which contains a segmentation network and two edge segmentation discriminators. The segmentation network, combined with an edge-aware self-supervision mechanism, is devised to conduct vessel segmentation with limited labeled patches. Meanwhile, two discriminators are introduced to distinguish whether the segmented vessel and background contain residual features in an adversarial manner. In the training stage, two discriminators are devised tocompete for the predicted position of edges. Exhaustive experiments demonstrate that, with only limited labeled patches, EVS-Net achieves a close performance of fully supervised methods, which provides a convenient tool for the pathological liver vessel segmentation. Code is publicly available at https://github.com/zju-vipa/EVS-Net.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 3

page 6

page 7

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

Introduction

Liver cancer is one of the malignant tumors with the highest mortality rate, which witnesses an increasing number of cases in recent years  d2017management. Hepatocellular carcinoma represents the most frequent primary liver tumor, taking up about of all liver cancer cases  perumpail2017clinical. The microvascular invasion (MVI) is a major prognostic factor in hepatocellular carcinoma, which significantly affects the choice of treatment and the overall survival of patients sumie2008microvascular; rodriguezperalvarez2013a

. However, the current MVI detection heavily depends on the experiences of doctors, which is a largely subjective and time-consuming to diagnose. Existing machine learning algorithms for predicting MVI mainly focused on computed tomography, magnetic resonance imaging, and positron emission tomography 

cuccurullo2018microvascular, yet ignore pathological images. Even to data, there is no learning-based method yet for automatically detecting MVI in pathological liver images.

Figure 1:

The overview of the edge-competing mechanism. The Over-Segmentation Discriminator (OS-D) and Under-Segmentation Discriminator (US-D) are devised for distinguishing whether the segmented background and segmented vessel contain residual features. The over-segmented and under-segmented patches are generated with different erosion and dilation operations following binarization processing. Only when the predicted mask depicts the precise segmentation will the true criterion of the two discriminators be satisfied simultaneously for an input patch.

As deep learning approaches have achieved promising results in the medical image analysis area 

ker2018deep; altaf2019going, we expect they have the potential to be deployed in the pathological liver vessel segmentation. There are many deep learning based segmentation methods for general natural images minaee2020image and medical images ronneberger2015u-net:; xue2018segan:; Dai2019Transfer. Due to the specific attributes of different images, however, existing methods cannot be directly applied to the pathological liver vessel segmentation. Moreover, the success of deep learning are highly based on a large number of annotated samples; yet for pathological liver vessel segmentation, there was not a dedicated dataset available, as it is extremely tedious, time-consuming, and expensive to annotate those vessels accurately.

In this paper, we first collect a pathological liver image dataset containing whole slide images with labels of vessels, MVI, and hepatocellular carcinoma grades. It can readily serve as a benchmark for the research on the analysis of pathological liver images. Fig. 2 shows some cut patches and the whole slide pathological image, which has a size of . For the patches, the coarse edges of most vessels can be obtained by binarization processing. For the vessels containing hepatocellular carcinoma cells, binarization processing will fail to segment those patches. However, combining with the erosion and dilation operation, binarization processing may easily produce some over-segmented or under-segmented vessels.

Based on such findings, we devise an edge-competing mechanism to assist the vessel segmentation, for which the overview is shown in Fig. 1. With the easily obtained over-segmented patches as the positive training samples, the over-segmentation discriminator is devised for distinguishing whether the segmented background contains residual features of vessels. Similarly, the under-segmentation discriminator is devised for distinguishing whether the segmented vessels contain residual features of the background. For an input patch, only when the predicted mask depicts the perfect segmentation will the true criterion of the two discriminators be satisfied simultaneously. It will effectively relieve the dependence of annotation by incorporating the edge-competing mechanism into the vessel segmentation framework.

Besides, from the whole slide image in Fig. 2, we can see that the pathological liver images have some unique attributes, such as super-large sample size, multi-scale vessel, blurred vessel edge, which is challenging for the accurate vessel segmentation. Given such specific characteristics of pathological liver images, we propose the Edge-competing Vessel Segmentation Network (EVS-Net), which is shown in Fig. 2. For the massive unlabeled patches, the doctor only needs to label limited patches as guidance. For the rest of the unlabeled patches, the binarization processing combined with erosion and dilation will generate many over-segmented and under-segmented patches.

As shown in Fig. 2, the proposed EVS-Net is composed of a Pathological Vessel Segmentation Network (PVSN), an Over-Segmentation Discriminator (OS-D), and an Under-Segmentation Discriminator (US-D). In the training stage, the limited labeled patches will supervise the PVSN to gain initial segmentation ability for the pathological vessel patches. The edge-competing mechanism is then incorporated into EVS-Net for assisting the training of the segmentation network in an adversarial manner.

To further strengthen consistency of the segmentation on massive unlabeled patches, an edge-aware self-supervision module is introduced to generate the self-supervision information for training PVSN. The core idea of edge-aware self-supervision is that, for the same image and segmentation network, the predicted mask of the transformed image should be equal to the transformed mask predicted by the network with the original image as input.

Our contribution can be summarized as follows. Firstly, we collect a pathological liver image dataset that contains whole slide images with labels of vessels, MVI, and hepatocellular carcinoma grades. It can readily serve as a benchmark for the analysis of hepatocellular carcinoma. Next, the proposed segmentation network combines an edge-aware self-supervision module and two segmentation discriminators to efficiently solve the challenges in the pathological vessel segmentation. Furthermore, the edge-competing mechanism between two discriminators is firstly proposed for learning accurate segmentation with over-segmented and under-segmented patches, which effectively relieves the dependence on a large number of annotations. Finally, with limited labels, the EVS-Net achieves results on par with fully supervised ones, providing a handy tool for the pathological liver vessel segmentation.

Figure 2: The flow diagram of pathological liver vessel segmentation. For massive unlabeled patches cut from the whole slide pathological image, the doctor only needs to label limited patches. Meanwhile, some under-segmented (over-seg.) patches and under-segmented (under-seg.) patches are generated through different erosion and dilation operations following binarization processing. Then, the EVS-Net is proposed for segmenting the vessels in the patches based on the above limited labels, over-segmented and under-segmented patches. The EVS-Net contains a Pathological Vessel Segmentation Network (PVSN), an over-segmentation discriminator (OS-D), and an under-segmentation discriminator (US-D). The PVSN is first initialized by training with the limited labels. With the over-segmented and under-segmented patches, the two discriminators are trained for distinguishing whether the segmented vessels and background contain residual features. In the training stage, for an unlabeled patch, only when the predicted mask is the perfect segmentation will the true criterion of the two discriminators be satisfied simultaneously, which facilitates the PVSN segmenting more accurately.

Related Works

To our knowledge, there are no vessel segmentation methods for pathological images until now. Existing methods, which are related to pathological images, mainly focus on the pathological grading of various carcinomas yang2019noninvasive; jiao2020a; liao2020d. Here, we give the related segmentation works for general natural images and medical images from two perspectives.

Edge-aware Segmentation. For edge-aware semantic segmentation, the commonly adopted framework is a two-branch network that simultaneously predicts segmentation maps and edges, where different constraints (bertasius2016semantic; chen2016semantic; heng2017fusionNet:; yu2018learning; Yin_2018_ECCV; Ye_2019_CVPR; takikawa2019gated-scnn:) are devised for strengthening the segmentation results with the predicted edges. Unlike predicting the edge directly, hayder2017boundary-aware proposed predicting pixels’ distance to the edge of object and post-processed the distance map into the final segmented results.  khoreva2017simple proposed edge-aware filtering to improve object delineation. zhang2017global-residual proposed a local edge refinement network to learn the position-adaptive propagation coefficients so that local contextual information from neighbors can be optimally captured for refining object edges. qin2019basnet: adopted the patch-level SSIM loss (wang2003multiscale) to assign higher weights to the edges. peng2017large proposed an edge refinement block to improve the localization performance near the object edges. Unlike the above methods, two edge-competing discriminators are devised for distinguishing whether the segmented vessels and background contain residual features. The over-segmented and under-segmented patches are adopted to train the two discriminators, focusing on the outer and inner area of edges.

GAN based Segmentation contains two categories: composition fidelity based methods ostyakov2018seigan:; remez2018learning; chen2019unsupervised; QiuECCV_2020 and mask distribution-based methods  luc2016semantic; han2017transferring; arbelle2018microscopy; hung2018adversarial; Ye_2020_CVPR. The composition fidelity based methods ostyakov2018seigan:; remez2018learning; chen2019unsupervised adopted the discriminator to distinguish the fidelity of natural images and the composited images, which is composition of the segmented objects and some background images. For the mask distribution-based methods, Luc et al. luc2016semantic adopted the adversarial optimization between segmented results and GT mask to train the segmentation network, which is the first GAN based semantic segmentation network. Next, some researchers han2017transferring; arbelle2018microscopy; xue2018segan: applied the same adversarial framework on the medical image segmentation task. Furthermore, Souly et al. souly2017semi and Hung et al. hung2018adversarial adopted the adversarial strategy to train the discriminator output class confidence maps. Unlike the above existing GAN-based methods, we devised two-discriminator architecture, where the two discriminators will compete the vessel edge’s position of the unlabeled patch in the adversarial training stage. In addition, the true inputs of the two discriminators are over-segmented and under-segmented patches.

Proposed Method

In the real diagnostic scenario, the assisting diagnosis tool of MVI should be objective, accurate, and convenient. The less time and labor cost, the better. Based on the real requirement of diagnosis, we introduce a pathological liver vessel segmentation method that can achieve applicable performance with limited labels. The proposed method contains two parts: the pathological patches pre-processing and pathological vessel segmentation. In the former part, the doctor only needs to label limited patches. Some over/under-segmented patches are generated with different erosion and dilation operations following binarization processing. In the latter part, the EVS-Net is devised for segmenting the pathological liver vessels based on the above limited labeled patches, over-segmented and under-segmented patches.

Pathological Patch Preprocessing

From Fig. 2, we can see that the pathological liver image has a super-large size and multi-scale vessel. Meanwhile, the areas of vessels and background are unbalanced. To get the useful training samples, we first locate the vessels’ general positions according to the white color. Then, three sizes are adopted to cut patches from the whole slide image around the located positions.

For the massive cut patch set , the doctor only needs to label limited patches . Next, for a patch , the under-segmented mask of patch x can be calculated as follows:

(1)

where, denotes the unit matrix that has the same size of the mask, denotes the OTSU binarization algorithm otsu1979a, denotes the disk structuring element. denotes the erosion operation by structure with radius , denotes the dilation operation by structure with radius . The and are integers. The obtained is the mask of background. The over-segmented patches constitute the over-segmented image set .

Similarly, the under-segmented image set is calculated as follows:

(2)

Pathological Vessel Segmentation

As shown in Fig. 2, the proposed EVS-Net is composed of a segmentation network and two discriminators. The segmentation network is trained with supervision loss on the labeled limited patches and an edge-aware self-supervision loss on unlabeled patches. The two discriminators are trained on over-segmented and under-segmented patches for distinguishing whether the segmented background and vessel contain residual features, respectively. In the adversarial training stage, the edge competition between two discriminators will facilitate the vessels’ accurate segmentation.

Segmentation Network.

In the EVS-Net, the segmentation network is designed to be an encoder-decoder architecture. With a labeled patch , the segmented result is expected to approximate the GT mask m, which can be achieved by minimizing the pixel-wise two-class Dice loss :

(3)

where, is the Laplace smoothing parameter for preventing zero error and reducing overfitting. For the labeled samples, we augment samples’ diversity through compositing background. The composited background is generated by adding other pure background and normalizing the new values into the normal range.

Edge-aware Self-supervision. To reduce the dependence on massive labeled samples, inspired by  wang2019self-supervised, we introduce an edge-aware self-supervision strategy, which can strengthen the edge consistency of vessels. The core idea is that the predicted mask of a transformed image should be equal to the transformed mask predicted by the network with the original image as input. Formally, for the robust segmentation network, given an affine transformation matrix M, segmented result of the affine transformed image should be consistent with the affine transformed mask in the following way: . Furthermore, we obtain the edge neighborhood weight map w as follows:

(4)

where, denotes the radius of the structure . With the weight map w, the edge-aware self-supervision loss is defined as follows:

(5)

where, and w are the weight maps of the predicted masks and , respectively. The edge-aware self-supervision mechanism not only strengthens the edge consistency of the segmented vessels but also can eliminate the unreasonable holes in the predict masks. The root reason is that unreasonable holes will be assigned high weights.

Edge-competing Discriminator.

Based on the obtained under-segmented set and under-segmented set , we devised two discriminators for distinguishing whether the segmented patch is a pure vessel or a pure background.

Over-segmentation Discriminator. For the over-segmented patch pairs , the background is computed using the following equation: , where ‘’ denotes pixel-wise multiplication. Then the concatenated triplet is set as the positive sample of OS-D. The true condition of OS-D is that the segmented background doesn’t contain any residual vessels’ features. For the unlabeled patch , the segmentation network predicts the mask of vessel . The reversed mask is calculated as follows: , where the denotes the unit matrix that has the same size of the mask. Next, the segmented background is computed using the following equation: , Then the concatenated triplet is fed to the OS-D , which discriminates whether the segmented background contains the vessels’ residual features. The is regarded as a false triplet. The adversarial optimization between the segmentation network and OS-D will constrain the segmented background predicted by the segmentation network don’t contain vessels’ features. The over-segmentation adversarial loss is given as follows:

(6)

where, the , are the generated background triplet distribution and under-segmented triplet distribution, respectively. The is sampled uniformly along straight lines between pairs of points sampled from the generated background triplet distribution and the under-segmented triplet distribution . The , where the is a random number between and . The second term (gradient penalty) is firstly proposed in WGAN-GP gulrajani2017improved. The is the gradient penalty coefficient.

Under-segmentation Discriminator. For the under-segmented patch pairs , the segmented vessel is calculated with the following equation: . Then the concatenated triplet satisfies the true condition of the US-D that the segmented vessel doesn’t contain any residual background features. For the unlabeled patch , the concatenated triplet is fed to the US-D , which distinguishes whether the segmented vessel contains the background residual features. The is regarded as the false triplet for US-D. The under-segmentation adversarial loss is given as follows:

(7)

where, the , are the generated vessel triplet distribution and under-segmented triplet distribution, respectively. The is sampled uniformly along straight lines between pairs of points sampled from the generated vessel triplet distribution and the under-segmented triplet distribution . The , where the is a random number between and . The optimization on will constrain the segmented vessel predicted by the segmentation network don’t contain background features.

Complete Algorithm

To sum up, the supervised Dice loss is adopted to train the pathological vessel segmentation network with the limited labeled patches. Then, edge-aware self-supervision is introduced to strengthen the edge consistency of the segmented mask with massive unlabeled patches. For the over-segmented patches and under-segmented patches, the edge adversarial losses and are adopted to train the OS-D and US-D. In the training stage, the two discriminators compete for the edge’s position of the predicted mask, which will refine the segmented mask. During training, we alternatively optimize the segmentation network and two edge discriminators using the randomly sampled samples from unlabeled patch set S, labeled patch set , under-segmented set , and under-segmented set . The complete algorithm is summarized in Algorithm 1.

1: The gradient penalty coefficient , interval iteration number , the batch size , the Laplace smoothing parameter

, Adam hyperparameters

, the balance parameters for and .
2: Initial critic parameters , initial segmentation networt parameters .
3:while  has not converged do
4:     for  do
5:         for  do
6:              Sample , , ,
7:     a random number .
8:              Obtain real triplet and .
9:              Obtain false triplet and .
10:              
11:         .
12:              
13:         .          
14:         .
15:         .      
16:     Sample unlabeled batch from S and
17:  labeled batch from .
18:     .
19:     .
20:     
21:   .
22:return Segmentation networt parameters , critic parameters .
Algorithm 1 The Training Algorithm for EVS-Net

Experiments

Dataset. The collected pathological liver image dataset contains whole slide images. Each image size has about . For each image, pathologists labeled the vessels’ masks, MVI, and hepatocellular carcinoma grades. There are total vessels where vessels contain hepatocellular carcinoma cells. The sample number for levels one, two, three and four tumor are , , , and , respectively. It can be a benchmark for the research on the analysis of pathological liver images. For all images, we finally collect patches, which have three scales (, and ). All patches are resized into . The train, validation, and test sample number are , , and , respectively.

Network architecture and parameters. The segmentation network we adopted is the DeeplabV3+ (backbone: resnet50 ) chen2017rethinking. In the experiment, the parameters are set as follows: , , the batch size , Adam hyperparameters for two discriminators . The radius and are random values between and . The radius equals to . The learning rate for the segmentation network and two discriminators are all set to be .

Metric. The metrics we adopted include Pixel Accuracy (PA), Mean Pixel Accuracy (MPA), Mean Intersection over Union (MIoU), Frequency Weighted Intersection over Union (FWIoU), and Dice.

Type Unsupervised Weakly Supervised Edge-aware Fully Supervised Ours
Method ReDO CAC ALSSS/ 500 USSS/ 500 G-SCNN BFP Unet FPN PSPNet PAN Deeplab E(100)

All patch

MPA 87.89 84.32 62.92/ 91.84 89.88/ 94.93 97.29 96.83 96.45 98.56 98.29 98.44 98.82 96.59
MIOU 82.20 78.35 55.41/ 89.40 84.57/ 92.35 97.28 96.04 92.76 97.40 96.86 97.11 97.79 94.38
FWIoU 89.34 80.57 74.45/ 93.77 90.75/ 95.46 95.89 96.32 95.64 98.46 98.13 98.29 98.69 96.67
DICE 96.56 88.32 91.99/ 98.08 97.02/ 98.58 98.76 98.21 98.62 99.52 99.42 99.47 99.60 98.96
1-14[0.8pt/2pt]

MVI patch

MPA 81.85 78.32 56.09/ 77.71 80.61/ 89.34 95.87 95.43 89.14 97.55 96.61 97.17 97.97 90.85
MIOU 74.06 69.87 48.21/ 72.43 73.82/ 84.83 92.85 94.72 81.77 95.60 94.45 94.82 95.84 86.19
FWIoU 84.75 79.10 71.55/ 84.44 84.81/ 91.24 95.90 94.53 89.16 97.48 96.80 97.00 97.60 91.98
DICE 94.95 85.32 91.28/ 95.19 95.09/ 97.25 94.83 97.53 96.42 99.23 99.01 99.07 99.29 97.46
Table 1: The quantitative results of different methods. Underline indicates the best performance among all methods. Bold indicates the best performance among all non-fully supervised methods. ‘(K)’ denotes EVS-Net with ‘K’ labeled patches. The ‘ALSSS/ ’ and ‘USSS/ ’ denote methods with or labeled patches (All scores are the average of three runs).
Figure 3: The visual result of different methods. The ‘’ denotes the method trained with labeled patches.

Quantitative Evaluation

To verify the effectiveness of EVS-Net, we compare the proposed method with the SOTA methods, including unsupervised methods (ReDO chen2019unsupervised, CAC hsu2018co-attention), weakly-/semi-supervised methods (ALSSS hung2018adversarial, USSS kalluri2019universal) and fully supervised methods (edge-aware methods:{ G-SCNN takikawa2019gated-scnn:, BFP ding2019boundary-aware},Unet ronneberger2015u-net:, FPN lin2017feature, PSPNet zhao2017pyramid, PAN li2018pyramid and DeeplabV3+ chen2017rethinking). For the weakly-/semi-supervised methods (USSS and ALSSS), we provide two groups of labeled patches ( patches and patches). The same data augmentation strategy is applied on the above labeled patches. Table 1 shows the quantitative results, where all the scores are average of three runs. For each trained model, we test it on all patches (‘All patch’) and patches (’MVI patch’) that contain hepatocellular carcinoma cells. It is obvious that DeeplabV3+ achieves the best performance among all the methods. The achieves SOTA results on par with existing non-fully supervised methods. Even with labeled samples, EVS-Net still achieves higher scores than non-fully supervised methods trained with hundreds of labeled patches on the ‘All patch’. In addition, even without labeled samples, (given in Table 2) still achieves better results than weakly/semi-supervised methods trained with labeled patches, which validates the effectiveness of the edge-competing mechanism. For ‘All patch’, has about increase on the scores of . Meanwhile, for ‘MVI patch’, has about performance increase compared with , which verifies the necessity of the limited labeled samples. More results of EVS-Net with different labeled patches are given in Table 2 and Fig. 4.

IndexAblation

All patch

MPA 95.73 94.28 93.59 90.90 94.77 95.47 96.24 96.38 96.72 96.59 98.48
MIoU 92.38 90.69 90.02 78.83 91.93 92.40 93.34 93.55 93.68 94.38 97.25
FWIoU 95.44 94.02 93.63 86.38 95.20 95.46 96.02 96.21 96.22 96.67 98.37
DICE 98.56 97.42 97.30 95.08 98.50 98.58 98.75 98.76 98.81 98.96 99.50
1-2[0.8pt/2pt] 3-6[0.8pt/2pt] 7-13[0.8pt/2pt]

MVI patch

MPA 87.80 88.42 87.26 80.47 85.12 87.89 89.81 89.63 89.25 90.85 97.16
MIoU 78.97 78.73 78.05 62.22 77.57 81.32 82.03 82.91 83.14 86.19 95.05
FWIoU 87.33 86.64 86.32 74.16 86.74 89.01 89.25 89.83 90.10 91.98 97.14
DICE 95.71 94.75 94.67 89.19 94.66 96.43 96.42 96.35 96.80 97.46 99.12
Table 2: The ablation study result on EVS-Net. , , , and denote the EVS-Net without edge-aware self-supervision, over-segmentation discriminator, under-segmentation discriminator, and two discriminators. denotes the EVS-Net with labeled patches. Italic indicates the best performance among methods trained with labeled patches. Bold and underline indicate the best and second-best performance among all methods (All scores in ).
Figure 4: The visual result of ablation study. denotes segmented vessels of method (Zooming in to compare edges).

Qualitative Evaluation

The visual results of different methods are shown in Fig. 3, where we can see that most of the fully supervised methods achieve similar results. Deep learning-based methods indeed have the potential to be applied to the pathological liver vessel segmentation. For the unsupervised methods (ReDO and CAC), the segmented background and vessels contain many small holes, which indicates the instability of the unsupervised methods. From the column , we can see that ALSSS with labeled patches fails to segment most cases, demonstrating that ten labeled patches cannot support enough supervision information for the semi-supervised method. Even with labeled patches, USSS still fails to segment the vessel with a large hepatocellular carcinoma area (row 4, column 8). For the blurry edge vessels (row 1 and row 5), EVS-Net with larger than labeled patches can achieve more accurate edge segmentation than non-fully supervised methods. For MVI patches, achieves inaccurate segmentation on those hepatocellular carcinoma areas, which indicates that limited labeled patches are necessity for guiding the segmentation of hepatocellular carcinoma areas. In summary, with limited labeled patches, EVS-Net can achieve close results on par with fully supervised methods.

Ablation Study

To verify the effectiveness of each component, we conduct an ablation study on the edge-aware self-supervision, over-segmentation discriminator, under-segmentation discriminator, two discriminators, and the different numbers of labeled patches. The , , and denote the EVS-Net without edge-aware self-supervision, over-segmentation discriminator, under-segmentation discriminator, and two discriminators, which are trained in the same setting of labeled patches. Table 2 shows the quantitative results of the above methods. The denotes the EVS-Net with labeled patches. We can see that achieves the best performance among , , and , which demonstrates that all components of EVS-Net are helpful for the segmentation performance. The performance of that doesn’t contain two discriminators degrades about compared with the . Methods with only one discriminator ( and ) reduce about compared with the . From Fig. 4, we can see that predicted masks of have accurate edges than and , which verifies the effectiveness and necessity of edge-competing. In addition, Fig. 4 shows that the result of has some holes. By contrast, the result of is accurate, which indicates that the edge-aware self-supervision is also beneficial for eliminating the incorrect holes. For the different numbers of labeled patches, we find that the critical cut-off point is -labeled-samples, which can supply relatively sufficient guidance. For the patches that have MVI, vessels’ edges are usually blurry, and vessels’ inner usually contain the large area of the hepatocellular carcinoma cells. Without any guidance of labeled samples, the pathological vessel segmentation network will fail to obtain the vessels’ desired edges. Therefore, the achieves the worst performance among all methods, which also verifies the necessity of the limited labeled patches. In addition, the comparisons between the segmented vessels of , , are shown in the last three columns on the right of Fig. 4, where we can see that segmented vessels’ contours of and contract inward and expand outward, respectively. With the edge-competing between OS-D and US-D, have accurate edges’ contours, which effectively clarifies the effectiveness of the edge-competing mechanism.

Conclusion

In this paper, we propose the first computer-aided diagnosis method for the pathological liver vessel segmentation, which is an essential and critical step for the diagnosis of MVI. For massive unlabeled pathological patches, the doctor only needs to label limited patches. Next, binarization processing combining with erosion and dilation is adopted to generate some under-segmented and under-segmented masks. Then, the EVS-Net is proposed for segmenting the pathological liver vessel with limited labels. The EVS-Net contains a pathological vessel segmentation network and two discriminators. With the limited labeled patches, the segmentation network is initialized with the supervision training. Meanwhile, an edge-aware self-supervision module is proposed for enhancing the edge consistency of massive unlabeled patches. With the over-segmented and under-segmented patches, two discriminators are devised for distinguishing whether the segmented vessels and background contain residual features. The segmentation network and two discriminators are trained in an adversarial manner. The edge-competing mechanism is verified to be very effective for facilitating the segmentation network segmenting more accurately. Exhaustive experiments demonstrate that the proposed EVS-Net achieves a close performance of fully supervised methods with limited labeled patches. It provides a convenient tool for the pathological liver vessel segmentation. In future work, we will focus on detecting and counting the hepatocellular carcinoma cells in the segmented vessels, which will provide a complete assisting diagnosis method for the diagnosis of MVI.

Acknowledgments

This work was supported by National Natural Science Foundation of China (No.62002318), Zhejiang Provincial Natural Science Foundation of China (LQ21F020003), Zhejiang Provincial Science and Technology Project for Public Welfare (LGF21F020020), Programs Supported by Ningbo Natural Science Foundation (202003N4318), and Alibaba-Zhejiang University Joint Research Institute of Frontier Technologies.

References