Log In Sign Up

Hypergraph Learning for Identification of COVID-19 with CT Imaging

by   Donglin Di, et al.

The coronavirus disease, named COVID-19, has become the largest global public health crisis since it started in early 2020. CT imaging has been used as a complementary tool to assist early screening, especially for the rapid identification of COVID-19 cases from community acquired pneumonia (CAP) cases. The main challenge in early screening is how to model the confusing cases in the COVID-19 and CAP groups, with very similar clinical manifestations and imaging features. To tackle this challenge, we propose an Uncertainty Vertex-weighted Hypergraph Learning (UVHL) method to identify COVID-19 from CAP using CT images. In particular, multiple types of features (including regional features and radiomics features) are first extracted from CT image for each case. Then, the relationship among different cases is formulated by a hypergraph structure, with each case represented as a vertex in the hypergraph. The uncertainty of each vertex is further computed with an uncertainty score measurement and used as a weight in the hypergraph. Finally, a learning process of the vertex-weighted hypergraph is used to predict whether a new testing case belongs to COVID-19 or not. Experiments on a large multi-center pneumonia dataset, consisting of 2,148 COVID-19 cases and 1,182 CAP cases from five hospitals, are conducted to evaluate the performance of the proposed method. Results demonstrate the effectiveness and robustness of our proposed method on the identification of COVID-19 in comparison to state-of-the-art methods.


page 1

page 3

page 5

page 6

page 7

page 9

page 10

page 11


Deep Learning System to Screen Coronavirus Disease 2019 Pneumonia

We found that the real time reverse transcription-polymerase chain react...

CovidCTNet: An Open-Source Deep Learning Approach to Identify Covid-19 Using CT Image

Coronavirus disease 2019 (Covid-19) is highly contagious with limited tr...

Large-Scale Screening of COVID-19 from Community Acquired Pneumonia using Infection Size-Aware Classification

The worldwide spread of coronavirus disease (COVID-19) has become a thre...

An automatic COVID-19 CT segmentation based on U-Net with attention mechanism

The coronavirus disease (COVID-19) pandemic has led a devastating effect...

Dual-Sampling Attention Network for Diagnosis of COVID-19 from Community Acquired Pneumonia

The coronavirus disease (COVID-19) is rapidly spreading all over the wor...

Contrastive Cross-site Learning with Redesigned Net for COVID-19 CT Classification

The pandemic of coronavirus disease 2019 (COVID-19) has lead to a global...

1 Introduction

The coronavirus disease pandemic, named COVID-19, has become the largest global public health crisis since in started it early of 2020. COVID-19 was caused by a kind of savagely contagious virus, and could lead to acute respiratory distress and multiple organ failure (Li et al., 2020a; Chen et al., 2020; Li et al., 2020b; Wang et al., 2020a; Holshue et al., 2020).

The latest guideline, published by the Chinese government (the trial sixth version) (of National Health Committee and others, ), declares that the diagnosis of COVID-19 must be confirmed by the reverse transcription polymerase chain reaction (RT-PCR) or gene sequencing for respiratory or blood specimens. Recent studies (Fang et al., 2020; Gozes et al., 2020; Xie et al., 2020) have investigated the sensitivity of non-contrast chest CT, and demonstrated that, recognizing either diffusion or focal ground-glass opacities as the disease characteristics in CT is a reliable and efficient approach. More specifically, the bilateral and peripheral ground-class and consolidative pulmonary opacities in CT are the typical features of COVID-19 symptoms, and the greater severity of the disease with increasing time from onset symptoms shows larger lung involvement and more linear opacities, a.k.a. the “crazy-paving” pattern and the “reverse halo” sign (Xie et al., 2020; Bernheim et al., 2020).

To reduce the workload in diagnosing COVID-19, plenty of machine learning-based studies have been conducted

(Gozes et al., 2020; Li et al., 2020a; Narin et al., 2020; Zhang et al., 2020; Shan+ et al., 2020). However, there are still two major challenges: 1) Noisy data, due to the large variations of data collected in an emergent situation, such as using different reconstruction kernels and CT manufactures, along with possible patient movements; 2) Confusing cases, due to similar radiological appearance of COVID-19 and other pneumonia, especially in the early stage. Therefore, how to handle these challenges is the key for successful identification of COVID-19 from CAP.

Accordingly, in this work, we propose an uncertainty based learning framework, called Uncertainty Vertex-weighted Hypergraph Learning (UVHL), to identify COVID-19 from CAP with CT images. The most essential task is to exploit the latent relationship among various COVID-19 cases and CAP cases, and then make a prediction for a new testing case, i.e., whether belonging to COVID-19 or not. The proposed framework employs a vertex-weighted hypergraph structure to formulate data correlation among different cases. The module of “uncertainty score measurement” is used to generate two metrics, i.e., 1) noisy data aleatoric uncertainty and 2) the model’s inability epistemic uncertainty. Then, the proposed UVHL conducts learning on the hypergraph structure to make a prediction for the new testing case, by simultaneously a) incorporating the uncertainty values of measured data to relieve the misleading patterns from noisy low-quality data and b)

allocating more attention to the nodes distributing around the classifying interface in the latent representation space. Another advantage of the proposed framework is its flexibility in utilizing multi-modal data/features when available. We apply our proposed method to a large dataset, with 2,148 COVID-19 cases and 1,182 CAP cases. The experimental results show that our proposed method can achieve a satisfactory accuracy of 90% for identification of COVID-19 from CAP.

The main contributions of this paper are summarized as follows:

  • We propose to formulate data correlation among all COVID-19 and CAP cases using hypergraph, for exploring high-order relationship using multi-type CT features (such as regional features and radiomics features).

  • We propose an uncertainty vertex-weighted strategy to relieve the influence of noisy (CT) data collected from suspected COVID-19 patients in emergent situation.

  • We have demonstrated better performance in the task of identifying COVID-19 from CAP, and have also shown how different types of CT features perform in this task.

Figure 1: Illustration of lung CT image, infection, lung lobes, and pulmonary segments on a CAP case (left) and a COVID-19 case (right).

2 Related Work

In this section, we briefly review recent works on diagnosing COVID-19 and introduce current studies on hypergraph learning.

2.1 AI-based COVID-19 Diagnosis

As introduced in (Zu et al., 2020), COVID-19 patients could be divided into mild, moderate, severe and critically ill stages, according to the severity of disease development. In the mild stage, the pneumonia symptom is difficult to be observed from CT images for a suspected patient. With the development of the disease, ground-glass opacity (GGO), increased crazy-paving pattern, and consolidation can be observed (Li and Xia, 2020). When it becomes a serious situation, the symptom will deteriorate and also the gradual resolution of consolidation could be observed in CT images.

In the very early studies, several statistics-based methods (Chen et al., 2020; Li et al., 2020b; Wang et al., 2020a) are proposed to develop automatic detection and patient monitoring methods for diagnosis of COVID-19. However, only simple data statistics is employed in these methods, which limits the capability of diagnosing suspected patients when facing the challenge of noisy data and confusing cases.

To further improve the performance, a group of AI-based methods (Narin et al., 2020; Shan+ et al., 2020; Gozes et al., 2020) are proposed in the following. In Bernheim et al. (2020); Shan+ et al. (2020); Tang et al. (2020), reliable representations from CT are learned to represent the symptom of COVID-19. The co-relationship between chest CT and RT-PCR testing has also been investigated in COVID-19 (Ai et al., 2020; Fang et al., 2020; Xie et al., 2020). Gozes et al. (2020) introduce an AI-based automatic CT image analysis tool for detection, quantification, and tracking of coronavirus.

Although there have been plenty of works on AI-assisted COVID-19 diagnosis tools, the identification of COVID-19 from CAP has not fully investigated, which has become an important issue recently. In this task, Wang et al. (2020b) propose to classify the patches of infected lesions into COVID-19 or typical viral pneumonia using the modified and fine-tuned Inception migration-learning model with the pre-trained weights, in which the infection patches need to be manually labeled. Another issue is the correlation among the COVID-19 cases and the CAP cases, which is important to identify the category of a new testing case.

2.2 Preliminary on Hypergraph Learning

Hypergraph learning has been widely applied in many tasks, such as identifying non-random structure in structural connectivity of the cortical microcircuits (Dotko et al., 2016), identifying high-order brain connectome biomarkers for disease diagnosis (Zu et al., 2016), and studying the co-relationships between functional and structural connectome data (Munsell et al., 2016). Hypergraph learning was first introduced in (Zhou et al., 2007), in which each node represents one case, each hyperedge captures the correlation between each pair of nodes, and the learning process is conducted on a hypergraph as a propagation process. By this method, the transductive inference on hypergraph aims to minimize the label differences between vertices that are connected by more and stronger hyperedges. Then, the hypergraph learning is conducted as a label propagation process on the hypergraph to obtain the label projection matrix (Liu et al., 2017)

, or as a spectral clustering

(Li and Milenkovic, 2017).

Other applications of hypergraph learning include video object segmentation (Huang et al., 2009), images ranking (Huang et al., 2010), and landmark retrieval (Zhu et al., 2015). Hypergraph learning has the advantage of modeling high-order correlation modeling, but the reliability of different vertices on the hypergraph, also important to conduct accurate learning, has not been well investigated.

3 Materials and Preprocessing

In this section, we first introduce materials used in this work and image preprocessing steps. Then, multi-type features, including regional features and radiomics features from CT images are extracted.

Figure 2: Illustration of our proposed Uncertainty Vertex-weighted Hypergraph Learning (UVHL) method for COVID-19 identification.

3.1 Dataset

In this study, a total of 3,330 CT images were collected, including 2,148 from COVID-19 patients and the rest 1,182 from CAP patients. These images were provided by the Ruijin Hospital of Shanghai Jiao Tong University, Tongji Hospital of Huazhong University of Science and Technology, China-Japan Union Hospital of Jilin University, Hangzhou First People’s Hospital of Zhejiang University, and Shanghai Public Health Clinical Center of Fudan University. All the COVID-19 cases were confirmed as positive by RT-PCR and acquired from Jan. 9, 2020 to Feb. 14, 2020. CAP images were obtained from Jul. 30, 2018 to Feb. 22, 2020. The CT scanners used in this study include uCT 780 from UIH, Optima CT520, Discovery CT750, LightSpeed 16 from GE, Aquilion ONE from Toshiba, SOMATOMForce from Siemens, and SCENARIA from Hitachi. The CT protocol here includes: 120 kV, reconstructed CT thickness ranging from 0.625 to 2mm, and breath-hold at full inspiration. All images were de-identified before sending for analysis. This study was approved by the Institutional Review Board of participating institutes. Written informed consent was waived due to retrospective nature of the study.

3.2 Preprocessing

In this study, both regional and radiomics features are extracted from CT image for each patient. More specifically, we first perform segmentation of left / right lung, 5 lung lobes, and 18 pulmonary segments, as well as infected lesions by deep learning based network,

i.e., VB-Net, in a portal software (Shan+ et al., 2020), for each CT image.

To generate regional features, we calculate a dimension of

features for each patient, including histogram distribution, infected lesion counting numbers, the mean and variance grey values of lesion area, lesion surface area, and additional density and mass features,

etc. To generate radiomics features, radiomics computation is performed on the infected lesions and a dimension of for each patient is extracted, including the first-order intensity statistics and texture features such as gray level co-occurrence matrix (Shi et al., 2020). With the information on age and sex also included, the representations for each patient can be concatenated as overall.

4 The Method

In this section, we introduce our proposed Uncertainty Vertex-weighted Hypergraph Learning (UVHL) method for COVID-19 identification. Figure 2 shows in the framework of our proposed method, which is composed of three steps, i.e., 1) “Data Uncertainty Measurement”, 2) “Uncertainty-vertex Hypergraph modeling” and 3) “Uncertainty-vertex Hypergraph Learning”, respectively.

4.1 Data Uncertainty Measurement

As introduced before, the data quality may suffer from the unstable, noisy nature caused in the emergent situation. To overcall this limitation, it is important to identify the reliability of different cases during the learning processing. In this step, a data uncertainty measurement process is conducted to generate uncertainty scores for all cases used in the learning processing. Here, two types of uncertainty factors are calculated in our method.

  1. Aleatoric Uncertainty. The data is abnormal, noisy or collected by mistake with low quality.

  2. Epistemic Uncertainty. The features of these cases lie around the decision boundary that makes the distinguishing model under a serious challenge.

We will introduce how to calculate these uncertainty scores in details as below.

4.1.1 Aleatoric Uncertainty

The aleatoric uncertainty represents the measure of the quality measure of noisy data, and is based on the comparison of data distributions. The objective is to estimate

that minimizes the Kullback-Leibler (KL) divergence between true distribution and predicted distribution over training samples:


Hence, the loss function can be defined as KL-Divergence:

, which is minimized during the training process. In detail, the loss for a single case can be calculated as Eq. 2:


where denotes the Cross-Entropy function,

denotes the feature vector of each patient,

is the label, and represents the network with softmax function as the last layer that maps features to the corresponding binary prediction. stands for the entropy of . denotes the predicted variance. To avoid the potential division by zero, we replace by . Therefore, can be used to predict the uncertainty score for each case.

Note that and are redundant for optimization. Therefore, for samples, we can rewrite the loss function as Eq. 3:


If the Cross-Entropy between the predicted and true label is quite large, the model tends to predict a higher to make inputs with high uncertainty having a smaller effect on the loss. This allows the network to learn to attenuate the effect from erroneous labels, thus becoming more robust to noisy data. In our task, we denote as aleatoric uncertainty to identify low quality data, as defined in Eq. 4:


4.1.2 Epistemic Uncertainty

Epistemic uncertainty refers to the model’s inability for accurate and precise prediction. To compute this measurement, we use the dropout variation inference, which is a widely adopted practical approach for approximate inference (Gal and Ghahramani, 2016). The Monte Carlo estimation method is referred as MC dropout. Our approximate predictive distribution is given by Eq. 5:



is a set of random variables for a model with

layers. and

denote the input and the corresponding output of any MC dropout model, respectively. The effect of our MC dropout can be attributed to impose a Gaussian distribution on each layer during the test stage. In detail, the multi-layer perception neural network (MLP) model can be trained with dropout. But different from the conventional settings, these dropout layers are kept open during the testing stage. Each case is predicted for

times, and the epistemic uncertainty for this case can be calculated using the variance of these values.

Therefore, the predicted result for one case can be obtained by Eq. 6:


or more specifically by Eq. 7 in our task:


Combined with aleatoric uncertainty introduced before, the epistemic uncertainty can be approximated as (Kendall and Gal, 2017) in Eq. 8:


where denotes the sample and denotes the test with dropout.

Note that (epistemic uncertainty) is mainly composed of aleatoric uncertainty. Consequently, when gets higher, it mainly represents lower data quality instead of the limitation on classification capability.

To normalize the epistemic uncertainty

, its mean and standard deviation in the whole dataset can be calculated as

. Then, sigmoid function is adopted to ensure the uncertainty score ranging from 0 to 1. is an adjustable parameter, to make different uncertainty cases more distinctive. If the is set to positive, the cases with the high uncertainty score will be adjusted higher, the cases with the low uncertainty score will be lower, and vice versa. Weights of all data are shown in Eq. 9:


In the end of this step, by leveraging the uncertainty, the quality of data is measured and also the weighted vertices are generated accordingly.

4.2 Uncertainty-vertex Hypergraph Construction

To identify the COVID-19 cases, it is important to exploit the data correlation. Here, the hypergraph structure is employed to model the relationship among the known training COVID-19 cases, the known training CAP cases, and the unknown testing cases.

In the hypergraph , each vertex denotes one case, and there are totally vertices according to the number of cases involved. Given the two types of features, i.e., the regional features and radiomics features, two groups of hyperedges are generated to build the connections among these cases. For the regional features, each time one vertex (case) is selected as the centroid, and its nearest neighbors (cases) are selected to be connected by one hyperedge. This process repeats until all vertices have been selected once. Then, a group of hyperedges based on the regional feature can be generated. The same process is performed for the radiomics feature, which generates another group of hyperedges. These two groups of hyperedges are concatenated to build the final hypergraph.

Different from conventional hypergraph, the uncertainty-vertex hypergraph not only cares about features and the label of each vertex, but also considers the uncertainty of each vertex. In this way, these more reliable vertices could contribute more during the learning process, and vice versa. Here, is the vertex set, is the hyperedges set, and is the pre-defined matrix of hyperedge weights. Besides these, denotes the uncertainty matrix for all the vertices. Therefore, our uncertainty-vertex hypergraph can be written as . Leveraging vertex weights , an incidence matrix is then generated to represent the relationship among different vertices.


In the end of this stage, the uncertainty vertex-weighted hypergraph is constructed to represent the correlation among all cases.

Figure 3: Besides the hyperedge weights, the uncertainty-vertex hypergraph contains the uncertainty score of each vertex.

4.3 Uncertainty-vertex Hypergraph Learning

As shown in Fig. 3, compared with the conventional hypergraph learning method, the proposed UVHL structure considers the uncertainty of each vertex individually and the learning process is conducted on an unequal space. The learning task on the uncertainty-vertex hypergraph can be formulated as:


More specifically, the smoothness regularizer function and the empirical loss term can be, respectively, rewritten as follows:


where is the column of and . The uncertainty vertex-weighted hypergraph loss function can be further rewritten as:


Therefore, the target label matrix can be obtained as:


With the generated label matrix ( in our task), the new coming testing case can be identified as COVID-19 or CAP accordingly.

5 Experiments

Classify as COVID-19 Classify as CAP
COVID-19 True Positive (TP) False Negative (FN)
CAP False Positive (FP) True Negative (TN)
Table 1:

The definition of the confusion matrix for COVID-19 identification.

SVM (p-value) 0.84084 1.173e-7 0.85714 1.438e-6 0.81034 4.235e-3 0.83374 1.037e-4 0.89423 0.0498 0.75200 3.283e-6
MLP (p-value) 0.84685 4.917e-6 0.86175 1.082e-5 0.81897 0.0153 0.84036 2.349e-3 0.89904 0.0507 0.76000 8.777e-9
iHL (p-value) 0.85135 5.260e-7 0.86327 3.415e-4 0.83052 0.0332 0.84790 7.905e-3 0.90256 0.2367 0.76866 2.088e-8
tHL (p-value) 0.86486 3.533e-4 0.89191 2.851e-4 0.81743 4.559e-3 0.85467 0.0197 0.89898 0.2383 0.80547 7.071e-5
UVHL (std) 0.89790 0.93269 0.84000 0.88635 0.90654 0.88235
Table 2: Performance comparison of different methods on the pneumonia dataset. (“” denotes the significance testing, .)

5.1 Evaluation Metrics

In our experiments, six criteria are employed to evaluate the COVID-19 identification performance, and the definition of the confusion matrix is shown in Table 1.

  1. Accuracy (ACC): ACC measures the proportion of samples that are correctly classified. .

  2. Sensitivity (SEN): SEN

    measures the proportion of actual positives that are correctly identified as such. This metric is also called as “recall”, reflecting the misdiagnose proportion. In actual medical diagnostic application scenarios, this evaluation metric is more critical.


  3. Specificity (SPEC): SPEC measures the proportion of actual negatives that are correctly identified as such. It stands for the omission diagnose rate. .

  4. Balance (BAC): BAC is the mean value of SEN and SPEC. .

  5. Positive Predictive Value (PPV): PPV measures the proportion of detected positives that are true positive. .

  6. Negative Predictive Value (NPV): NPV measures the proportion of detected negatives that are true negative. .

5.2 Compared Methods

The following popular classification approaches are used for comparison:

  • Support Vector Machine (SVM) (Cortes and Vapnik, 1995)

    : It is a non-probabilistic linear classifier, used to perform supervised learning. It selects a group of the training data as support vectors to determine the boundary that divides different categories apart as unambiguously as possible.

  • Multilayer Perceptron (MLP) Neural Network: As the fundamental feed-forward artificial neural network, MLP can be utilized to perform binary classification with the cross-entropy as the loss function.

  • Inductive Hypergraph Learning (iHL) (Zhang et al., 2018): In iHL, all available features are combined into one single feature, and then a projection is learned on the hypergraph structure, which is used to conduct classification task on the pneumonia instances. This model learns the high-order representations from the training set and is evaluated in the testing set.

  • Transductive Hypergraph Learning (tHL) (Zhou et al., 2007)

    : The transductive learning on hypergraph is conducted to learn the label matrix. Both the training data and all testing data are employed in the hypergraph structure, yet leading to the commonly used semi-supervised learning approach.

Figure 4: The performance of UVHL and compared methods. The results show that UVHL outperforms other methods for all metrics.

5.3 Implementation

In our experiments, the whole dataset consists of 2,148 COVID-19 cases and 1,182 CAP cases.

We randomly divide them into 10 subsets and perform 10-fold cross-validation, in which 9 subfolds are used for training and the rest one is used for testing each time. The data splitting process repeats 10 times, and the mean and standard deviation of all 10 runs are reported as the final result for comparison. All features are normalized into in the training dataset, and the offset mean and variance are applied to the testing dataset for data normalization, respectively.

For our UVHL model, K nearest neighbors are connected for each vertex when generating hyperedges. We note that it is important to generate a suitable hypergraph structure for representation learning. However, how to select the best value in this procedure is difficult. A large will lead to high dissimilarity insider the hyperedge, while a small may be not informative enough to the overall hypergraph structure. To select a suitable , the following strategy is adopted to select . First, a pool of candidate values is set as in our experiments. Given a set of training data and corresponding testing data, we further split the training data into 5 folds. The 5-fold cross-validation is conducted on the training data, where different are used. We then collect the performance of different on the training data, and the with the best performance is used for testing. In this way, the selection of can be fully automatic and optimized.

5.4 Results and Discussions

Experimental results are demonstrated in Fig. 4, and the detailed mean value and the significance of the t-test between UVHL and other methods are listed in Table 2. From these results, we have the following observations:

  1. Our proposed method UVHL achieves the most reliable performance among all metrics. Compared with SVM and MLP, our approach obtains better performance (i.e., 6.79% and 6.03% relative improvement in terms of ACC, respectively), demonstrating that the hypergraph based approach has the effective ability to tackle the pneumonia identification task.

  2. Compared with other hypergraph based methods, i.e., inductive hypergraph learning (iHL) (Zhang et al., 2018) and transductive hypergraph learning (tHL) (Zhou et al., 2007), our approach achieves relative gains of 5.47% and 3.82% in terms of ACC, respectively.

  3. Besides the better sensitivity value, our proposed UVHL method achieves much higher specificity value compared with all other methods. This indicates that our proposed method can not only have high recall of COVID-19 patients but also be effective on filtering CAP patients, which is quite useful in practice.

Weighting strategy ACC SEN SPEC BAC PPV NPV
1 Equal Weight 0.85586 0.88426 0.80342 0.84384 0.89252 0.789912
2 Support Vectors 0.86066 0.87021 0.84442 0.85731 0.90983 0.78137
3 Aleatoric Uncertainty 0.87387 0.918919 0.78378 0.85135 0.89474 0.82857
4 Epistemic Uncertainty 0.88589 0.90741 0.84615 0.87678 0.91589 0.83193
5 Proposed Uncertainty 0.89790 0.93269 0.84000 0.88635 0.90654 0.88235
Table 3: Experimental comparison on the data uncertainty measurement.

5.5 Data Uncertainty Study

To evaluate the effectiveness of our proposed data uncertainty method, we further conduct ablation experiments to compare variants of the data uncertainty measurement procedure. First, we remove the uncertainty measurement procedure and treat all cases equally. Secondly, the SVM-based uncertainty score is calculated, instead of that of using MLP. Then, the two uncertainty measurements are used individually for comparison. Experimental results are reported in Table 3, from which we can have the following observations:

  1. Compared with the method without uncertainty, i.e., with equal weights, all the other methods with uncertainty can achieve better performance.

  2. The method with uncertainty from SVM performs worse than that of using MLP. It indicates that MLP has better identification effectiveness compared with SVM on uncertainty measurement.

  3. Compared with the case of using aleatoric uncertainty and epistemic uncertainty individually, the use of both uncertainties, i.e., the proposed method, achieves the best performance, which demonstrates the effectiveness of our proposed data uncertainty strategy.

5.6 Analysis On Feature Types

In this study, there are two types of features from CT, i.e., regional features and radiomics features. Here, we evaluate the effectiveness of these two features on the task of COVID-19 identification. We have conducted experiments with our proposed method using each feature individually. Experimental comparison is demonstrated in Table 4. Our method using regional feature has higher sensitivity, while the specificity is relatively lower, compared with the cases of using radiomics features. These results indicate that regional feature is better in finding the true positive COVID-19 cases, while radiomics features have the advantage of identifying CAP cases. When using both types of features in our proposed method, the performance becomes stable, along with both increasing sensitivity and specificity, as shown in the last row of Table 4. This observation demonstrates that our proposed method has the ability of jointly utilizing multi-type features and achieve better performance.

Figure 5: Performance comparison with respect to different scales of training data.
Regional 0.85886 0.90323 0.77586 0.83954 0.88288 0.81081
Radiomics 0.85946 0.86982 0.84182 0.85582 0.90889 0.78012
Both 0.89790 0.93269 0.84000 0.88635 0.90654 0.88235
Table 4: Experimental comparison on different feature types and their combination.

5.7 Analysis on Few Labeled Data

As the large-scale labeled data for COVID-19 is expensive and maybe infeasible in emergent situations, how these methods perform with very limited labeled data is an important issue. It should be noted that we have not included MLP, as MLP performs very badly when having very few training data. To do that, we investigate how the compared methods work with respect to a small scale of labeled data from 10 to 100 for COVID-19 and CAP respectively. In these experiments, 100 cases for each category are selected as the validation data. The training data selection process repeats 10 times and the average performance is calculated for comparison. Experimental results are shown in Fig. 5. As shown in these results, we can observe that SVM performs inferior in all settings when given just very few labeled data, and the hypergraph based methods perform the best. We can also observe that our proposed method, i.e., UVHL, can achieve very stable performance when only a few labeled data are available, which justifies the effectiveness of our proposed method in these difficult situations.

6 Conclusion

In this paper, we propose an uncertainty vertex-weighted hypergraph learning method to identify COVID-19 from CAP using CT images. Confronting the challenging issues from noisy data and confusing cases with similar clinical manifestations and imaging features, our proposed method employs a hypergraph structure to formulate the data correlation among the known COVID-19 cases, the known as CAP cases, and the testing cases. Through this method, two types of CT image features (including regional features and radiomics features) are extracted for patient representation. To overcome the limitations of the noisy data, a data uncertainty measurement process is conducted to measure the uncertainty of each training case. Finally, a vertex-weighted hypergraph learning process is used to predict whether a new case is COVID-19 or CAP. We have conducted experiments on a large multi-center pneumonia dataset, including 2,148 COVID-19 cases and 1,182 CAP cases from 5 hospitals, and the experimental results demonstrate the effectiveness of our proposed method on identification of COVID-19 in comparison to the existing state-of-the-art methods.

In future work, the effectiveness of each individual feature should be fully investigated. Regarding the limited data and possible evolution of COVID-19, it is important to explore small sample learning methods as well as transfer learning techniques on this difficult task of identifying COVID-19.


  • T. Ai, Z. Yang, H. Hou, C. Zhan, C. Chen, W. Lv, Q. Tao, Z. Sun, and L. Xia (2020) Correlation of chest ct and rt-pcr testing in coronavirus disease 2019 (covid-19) in china: a report of 1014 cases. Radiology, pp. 200642. Cited by: §2.1.
  • A. Bernheim, X. Mei, M. Huang, Y. Yang, Z. A. Fayad, N. Zhang, K. Diao, B. Lin, X. Zhu, K. Li, et al. (2020) Chest ct findings in coronavirus disease-19 (covid-19): relationship to duration of infection. Radiology, pp. 200463. Cited by: §1, §2.1.
  • N. Chen, M. Zhou, X. Dong, J. Qu, F. Gong, Y. Han, Y. Qiu, J. Wang, Y. Liu, Y. Wei, et al. (2020) Epidemiological and clinical characteristics of 99 cases of 2019 novel coronavirus pneumonia in wuhan, china: a descriptive study. The Lancet 395 (10223), pp. 507–513. Cited by: §1, §2.1.
  • C. Cortes and V. Vapnik (1995) Support-vector networks. Machine learning 20 (3), pp. 273–297. Cited by: item 1.
  • P. Dotko, K. Hess, R. Levi, M. Nolte, M. Reimann, M. Scolamiero, K. Turner, E. Muller, and H. Markram (2016) Topological analysis of the connectome of digital reconstructions of neural microcircuits. arXiv preprint arXiv:1601.01580. Cited by: §2.2.
  • Y. Fang, H. Zhang, J. Xie, M. Lin, L. Ying, P. Pang, and W. Ji (2020) Sensitivity of chest ct for covid-19: comparison to rt-pcr. Radiology, pp. 200432. Cited by: §1, §2.1.
  • Y. Gal and Z. Ghahramani (2016) Dropout as a bayesian approximation: representing model uncertainty in deep learning. In international conference on machine learning, pp. 1050–1059. Cited by: §4.1.2.
  • O. Gozes, M. Frid-Adar, H. Greenspan, P. D. Browning, H. Zhang, W. Ji, A. Bernheim, and E. Siegel (2020) Rapid ai development cycle for the coronavirus (covid-19) pandemic: initial results for automated detection & patient monitoring using deep learning ct image analysis. arXiv preprint arXiv:2003.05037. Cited by: §1, §1, §2.1.
  • M. L. Holshue, C. DeBolt, S. Lindquist, K. H. Lofy, J. Wiesman, H. Bruce, C. Spitters, K. Ericson, S. Wilkerson, A. Tural, et al. (2020) First case of 2019 novel coronavirus in the united states. New England Journal of Medicine. Cited by: §1.
  • Y. Huang, Q. Liu, and D. Metaxas (2009) ] Video object segmentation by hypergraph cut. In

    2009 IEEE conference on computer vision and pattern recognition

    pp. 1738–1745. Cited by: §2.2.
  • Y. Huang, Q. Liu, S. Zhang, and D. N. Metaxas (2010) Image retrieval via probabilistic hypergraph ranking. In 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 3376–3383. Cited by: §2.2.
  • A. Kendall and Y. Gal (2017) What uncertainties do we need in bayesian deep learning for computer vision?. In Advances in neural information processing systems, pp. 5574–5584. Cited by: §4.1.2.
  • L. Li, L. Qin, Z. Xu, Y. Yin, X. Wang, B. Kong, J. Bai, Y. Lu, Z. Fang, Q. Song, et al. (2020a) Artificial intelligence distinguishes covid-19 from community acquired pneumonia on chest ct. Radiology, pp. 200905. Cited by: §1, §1.
  • P. Li and O. Milenkovic (2017) Inhomogeneous hypergraph clustering with applications. In Advances in Neural Information Processing Systems, pp. 2308–2318. Cited by: §2.2.
  • Q. Li, X. Guan, P. Wu, X. Wang, L. Zhou, Y. Tong, R. Ren, K. S. Leung, E. H. Lau, J. Y. Wong, et al. (2020b) Early transmission dynamics in wuhan, china, of novel coronavirus–infected pneumonia. New England Journal of Medicine. Cited by: §1, §2.1.
  • Y. Li and L. Xia (2020) Coronavirus disease 2019 (covid-19): role of chest ct in diagnosis and management. American Journal of Roentgenology, pp. 1–7. Cited by: §2.1.
  • M. Liu, J. Zhang, P. Yap, and D. Shen (2017) View-aligned hypergraph learning for alzheimer’s disease diagnosis with incomplete multi-modality data. Medical image analysis 36, pp. 123–134. Cited by: §2.2.
  • B. C. Munsell, G. Wu, Y. Gao, N. Desisto, and M. Styner (2016) Identifying relationships in functional and structural connectome data using a hypergraph learning method. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 9–17. Cited by: §2.2.
  • A. Narin, C. Kaya, and Z. Pamuk (2020)

    Automatic detection of coronavirus disease (covid-19) using x-ray images and deep convolutional neural networks

    arXiv preprint arXiv:2003.10849. Cited by: §1, §2.1.
  • [20] G. O. of National Health Committee et al. Office of state administration of traditional chinese medicine. notice on the issuance of a programme for the diagnosis and treatment of novel coronavirus (2019-ncov) infected pneumonia (trial sixth edition). 2020. Cited by: §1.
  • F. Shan+, Y. Gao+, J. Wang, W. Shi, N. Shi, M. Han, Z. Xue, D. Shen, and Y. Shi (2020) Lung infection quantification of covid-19 in ct images with deep learning. arXiv preprint arXiv:2003.04655. Cited by: §1, §2.1, §3.2.
  • F. Shi, L. Xia, F. Shan, D. Wu, Y. Wei, H. Yuan, H. Jiang, Y. Gao, H. Sui, and D. Shen (2020) Large-scale screening of covid-19 from community acquired pneumonia using infection size-aware classification. arXiv preprint arXiv:2003.09860. Cited by: §3.2.
  • Z. Tang, W. Zhao, X. Xie, Z. Zhong, F. Shi, J. Liu, and D. Shen (2020) Severity assessment of coronavirus disease 2019 (covid-19) using quantitative features from chest ct images. arXiv preprint arXiv:2003.11988. Cited by: §2.1.
  • D. Wang, B. Hu, C. Hu, F. Zhu, X. Liu, J. Zhang, B. Wang, H. Xiang, Z. Cheng, Y. Xiong, et al. (2020a) Clinical characteristics of 138 hospitalized patients with 2019 novel coronavirus–infected pneumonia in wuhan, china. Jama. Cited by: §1, §2.1.
  • S. Wang, B. Kang, J. Ma, X. Zeng, M. Xiao, J. Guo, M. Cai, J. Yang, Y. Li, X. Meng, et al. (2020b) A deep learning algorithm using ct images to screen for corona virus disease (covid-19). medRxiv. Cited by: §2.1.
  • X. Xie, Z. Zhong, W. Zhao, C. Zheng, F. Wang, and J. Liu (2020) Chest ct for typical 2019-ncov pneumonia: relationship to negative rt-pcr testing. Radiology, pp. 200343. Cited by: §1, §2.1.
  • J. Zhang, Y. Xie, Y. Li, C. Shen, and Y. Xia (2020)

    COVID-19 screening on chest x-ray images using deep learning based anomaly detection

    arXiv preprint arXiv:2003.12338. Cited by: §1.
  • Z. Zhang, H. Lin, X. Zhao, R. Ji, and Y. Gao (2018) Inductive multi-hypergraph learning and its application on view-based 3d object classification. IEEE Transactions on Image Processing 27 (12), pp. 5957–5968. Cited by: item 3, item 2.
  • D. Zhou, J. Huang, and B. Schölkopf (2007) Learning with hypergraphs: clustering, classification, and embedding. In Advances in neural information processing systems, pp. 1601–1608. Cited by: §2.2, item 4, item 2.
  • L. Zhu, J. Shen, H. Jin, R. Zheng, and L. Xie (2015) Content-based visual landmark search via multimodal hypergraph learning. IEEE Transactions on Cybernetics 45 (12), pp. 2756–2769. Cited by: §2.2.
  • C. Zu, Y. Gao, B. Munsell, M. Kim, Z. Peng, Y. Zhu, W. Gao, D. Zhang, D. Shen, and G. Wu (2016) Identifying high order brain connectome biomarkers via learning on hypergraph. In International Workshop on Machine Learning in Medical Imaging, pp. 1–9. Cited by: §2.2.
  • Z. Y. Zu, M. D. Jiang, P. P. Xu, W. Chen, Q. Q. Ni, G. M. Lu, and L. J. Zhang (2020) Coronavirus disease 2019 (covid-19): a perspective from china. Radiology, pp. 200490. Cited by: §2.1.