Breast mass classification in ultrasound based on Kendall's shape manifold

by   Michał Byra, et al.
Polish Academy of Sciences

Morphological features play an important role in breast mass classification in sonography. While benign breast masses tend to have a well-defined ellipsoidal contour, shape of malignant breast masses is commonly ill-defined and highly variable. Various handcrafted morphological features have been developed over the years to assess this phenomenon and help the radiologists differentiate benign and malignant masses. In this paper we propose an automatic approach to morphology analysis, we express shapes of breast masses as points on the Kendall's shape manifold. Next, we use the full Procrustes distance to develop support vector machine classifiers for breast mass differentiation. The usefulness of our method is demonstrated using a dataset of B-mode images collected from 163 breast masses. Our method achieved area under the receiver operating characteristic curve of 0.81. The proposed method can be used to assess shapes of breast masses in ultrasound without any feature engineering.


Added value of morphological features to breast lesion diagnosis in ultrasound

Ultrasound imaging plays an important role in breast lesion differentiat...

Improving the diagnosis of breast cancer based on biophysical ultrasound features utilizing machine learning

The improved diagnostic accuracy of ultrasound breast examinations remai...

Breast Mass Segmentation and Shape Classification in Mammograms Using Deep Neural Networks

Mammogram analysis to manually extract breast masses is a tough assignme...

Textural Approach for Mass Abnormality Segmentation in Mammographic Images

Mass abnormality segmentation is a vital step for the medical diagnostic...

Comparing Results of Thermographic Images Based Diagnosis for Breast Diseases

This paper examines the potential contribution of infrared (IR) imaging ...

Deep Structured learning for mass segmentation from Mammograms

In this paper, we present a novel method for the segmentation of breast ...

I Introduction

Breast mass shape in 2D ultrasound (US) imaging is an important factor taken into account by radiologists to accurately differentiate malignant and benign breast masses. While benign masses tend to have well-defined ellipsoidal shape, malignant masses usually have variable contour, which includes lobulations and spiculations [1]. Assessment of breast mass shape is an important part of the Breast Imaging Reporting and Data System (BI-RADS), which was developed by American College of Radiology to standardize the reporting process and diagnosis of breast masses in US. To help the radiologists assess breast masses in US various computer aided diagnosis systems have been proposed [2]. Those system commonly aim to computerize shape related information to extract morphological features that would be useful for the classification [3, 4, 5]. For example, features related to breast mass width-to-depth ratio, circularity and boundary roughness were reported to be efficient for the differentiation of malignant and benign breast masses [2].

In this work we propose a novel approach to breast mass shape analysis in US. Our approach does not require feature engineering, therefore is more automatic than the previous methods based on the handcrafted morphological features. To assess breast mass shape we utilize Kendall’s shape manifold [6]. In Kendall’s formalism, shape is related to geometrical properties of a 2D object, which are invariant to rotation, scale and translation. Shape of any object can be represented as a set of landmarks, points on object contour that capture shape variability. This set of landmarks corresponds to a point on the shape manifold. Similarly, shape of a breast mass can be expressed as a point on the Kendall’s shape manifold using landmarks. Next, by defining a distance function between points on the manifold it is possible to develop a classification method [7, 8]. Shapes of different objects can be compared without engineering features that would accurately describe geometrical properties of those objects. For example, the Procrustes distance function can be used to develop a support vector machine (SVM) classifier that can differentiate 2D shapes specified by landmarks [7]. As far as we know, this is the first application of the Kendall’s shape theory in US imaging. Image recognition methods based on the Kendall’s shape manifold were employed, for example, for plant species classification [7, 8]. Here, we use a similar approach to the one proposed in [7] to address the problem of breast mass classification. We express contours of breast masses as sets of landmarks and develop SVM classifiers that can directly differentiate malignant and benign masses.

Ii Materials and Methods

Ii-a Dataset

To investigate the usefulness of the Kendall’s shape manifold for breast mass classification we used a publicly available dataset of US breast mass images named UDIAT [9]. This dataset includes 163 US images collected from 53 malignant and 110 benign breast masses, respectively. The dataset was originally used to develop algorithms for breast mass detection and segmentation [9, 10]. In the case of the UDIAT dataset each B-mode image comes with a region of interest (ROI) indicating breast mass position (Fig. 1).

Fig. 1: Ultrasound B-mode images corresponding to a) malignant mass and b) benign mass, and the corresponding manual segmentations indicating breast mass shapes. Contours of malignant masses are commonly more variable than for benign cases.

Ii-B Shape analysis

Contour of a 2D shape can be represented by a finite set of landmarks, specific and unique points on contour. Each landmark is coded by a single complex number, with the real and imaginary parts corresponding to axial and lateral landmark coordinates, therefore for landmarks shape of an object is initially represented as a -dimensional complex vector [6]. Fig. 2 presents a breast mass with landmarks indicating points on contour. In this case, the distances between the consecutive points measured along the contour were the same. Next, to make the landmark shape representation invariant to scaling and translation, the complex vector is processed to have zero mean and unit norm. This shape representation, after the processing, is dubbed pre-shape, and denoted by . Additionally, to remove rotation different rotated versions of pre-shape are determined and final shape corresponds to the equivalence class of pre-shapes, denoted by [7].

The similarity of two shapes and can be assessed with the full Procrustes (FP) distance:


where and are the corresponding pre-shapes, and

denote the inner product and complex number norm, respectively. The FP distance can be used to define a positive definite kernel function based on the Gaussian radial basis function



where is the kernel parameter. Points on the shape manifold can be mapped to elements in a high dimensional Hilbert space. This Hilbert space embedding of shapes can be utilized to develop a binary SVM classifier. Thanks to the kernel trick, the SVM classifier can be expressed in the following dual form [11]:


where is the number of training samples, are the weights corresponding to each sample, are the corresponding sample classes (benign or malignant in our case) and is the kernel function from eq. 2, which is a valid kernel function for the SVM algorithm [7]. The samples with non-zero are denoted support vectors.

Fig. 2: Ultrasound B-mode image of a benign mass with 18 equidistant landmarks indicating shape.

Ii-C Statistical analysis

To evaluate the proposed method we applied leave-one-out cross-validation. The SVM classifiers were developed for breast mass shapes coded with different numbers of landmarks equal to 25, 50, 100 and 200, respectively. The classification performance was assessed using the receiver operating characteristic (ROC) curve and the area under the ROC curve. Accuracy, sensitivity and specificity of each classifier were calculated using the classifier cut-off value corresponding to the point on ROC curve, which was the closest to curve upper left corner, point (0, 1). Differences in AUC values were compared using the DeLong test implemented in R [12, 13]. Calculations, including implementation of the SVM classifier, were done in Matlab (Mathworks, MA).

Iii Results

Table 1 presents the breast mass classification performance results obtained for the proposed method. The SVM classifiers utilizing different number of landmarks achieved AUC values of around 0.81. There were no associated differences in the AUC values calculated for the SVM classifiers (-values0.05). The ROC curve obtained for the method based on 200 landmarks is presented in Fig. 3.

Fig. 3: The receiver operating characteristic (ROC) curve calculated for the proposed method (200 landmarks) utilizing Kendall’s shape manifold, with the area under the curve (AUC) equal to 0.81.
# Landmarks AUC Accuracy Sensitivity Specificity
25 0.810.03 0.810.2 0.700.05 0.740.04
50 0.810.03 0.760.1 0.740.04 0.750.04
100 0.810.03 0.760.1 0.780.04 0.740.03
200 0.810.03 0.760.1 0.780.04 0.740.04
TABLE I: Classification performance of the proposed method based on Kendall’s shape manifold obtained for different number of used landmarks. AUC - area under the receiver operating characterstic curve

Iv Discussion

In this work we presented the feasibility of using Kendall’s shape manifold for breast mass classification in US. Our approach, based on the Gaussian kernel with the FP distance and the SVM algorithm, achieved AUC value of 0.81, indicating medium classification performance. Our approach has several advantages. First of all, standard methods usually require expert knowledge to develop better performing morphological features. In comparison, the proposed approach is semi-automatic. In our case the problem of feature engineering is replaced with the task how to efficiently select landmarks and classification method. Currently, deep learning methods are gaining momentum for breast mass classification

[14, 15, 16, 17, 18, 19]. These deep learning methods can process US images fully automatically to extract image features related to malignancy and perform classification. While our method require to first manually segment each breast mass, in comparison to deep learning methods our approach does not require large dataset to develop the classifier.

There are also several issues related to our approach. First, our method, like the standard approaches based on morphological features, still depends on the quality of the initial manual segmentation. Moreover, the interpretability of the proposed machine learning method is lower than in the case of the methods based on expert knowledge. Morphological features related to shape properties, such as circularity or depth-to-width ratio, have clear interpretation. Nevertheless, for a test breast mass US image, our approach can be used to retrieve similar US images from the training set to help understand the classification decision. Another issue is related to the shape representation in Kendall’s formalism. This representation is invariant to scale and rotation, but those two properties might be related to breast mass malignancy [2].

In future we would like to compare the performance of our approach with a classifier developed using morphological handcrafted features. Moreover, in our work we used only one approach to shape analysis based on landmark representation, but it would be interesting to explore the usefulness of approaches based on other representations, for example those related to level sets [20].

V Conclusions

In this work we proposed a novel approach to breast mass differentiation in ultrasound. Our method, based on the Kendall’s shape manifold, achieved medium classification performance. The proposed approach can be used to assess breast mass shapes without any feature engineering.

Conflict of interest statement

The authors do not have any conflicts of interest.


  • [1] G. Rahbar, A. C. Sie, G. C. Hansen, J. S. Prince, M. L. Melany, H. E. Reynolds, V. P. Jackson, J. W. Sayre, and L. W. Bassett, “Benign versus malignant solid breast masses: Us differentiation,” Radiology, vol. 213, no. 3, pp. 889–894, 1999.
  • [2] W. G. Flores, W. C. de Albuquerque Pereira, and A. F. C. Infantosi, “Improving classification performance of breast lesions on ultrasonography,” Pattern Recognition, vol. 48, no. 4, pp. 1125–1136, 2015.
  • [3] A. V. Alvarenga, A. F. C. Infantosi, W. C. Pereira, and C. M. Azevedo, “Assessing the combined performance of texture and morphological parameters in distinguishing breast tumors in ultrasound images,” Medical physics, vol. 39, no. 12, pp. 7350–7358, 2012.
  • [4]

    C.-M. Chen, Y.-H. Chou, K.-C. Han, G.-S. Hung, C.-M. Tiu, H.-J. Chiou, and S.-Y. Chiou, “Breast lesions on sonograms: computer-aided diagnosis with nearly setting-independent features and artificial neural networks,”

    Radiology, vol. 226, no. 2, pp. 504–514, 2003.
  • [5]

    Y.-H. Chou, C.-M. Tiu, G.-S. Hung, S.-C. Wu, T. Y. Chang, and H. K. Chiang, “Stepwise logistic regression analysis of tumor contour features for breast ultrasound diagnosis,”

    Ultrasound in medicine & biology, vol. 27, no. 11, pp. 1493–1498, 2001.
  • [6] D. G. Kendall, “Shape manifolds, procrustean metrics, and complex projective spaces,” Bulletin of the London Mathematical Society, vol. 16, no. 2, pp. 81–121, 1984.
  • [7] S. Jayasumana, M. Salzmann, H. Li, and M. Harandi, “A framework for shape analysis via hilbert space embedding,” in

    Proceedings of the IEEE International Conference on Computer Vision

    , 2013, pp. 1249–1256.
  • [8] R. Mtibaa, E. Zagrouba et al., “Bayesian approach in kendall shape space for plant species classification,” in Iberian Conference on Pattern Recognition and Image Analysis.   Springer, 2017, pp. 322–331.
  • [9]

    M. H. Yap, G. Pons, J. Martí, S. Ganau, M. Sentís, R. Zwiggelaar, A. K. Davison, and R. Martí, “Automated breast ultrasound lesions detection using convolutional neural networks,”

    IEEE journal of biomedical and health informatics, vol. 22, no. 4, pp. 1218–1226, 2018.
  • [10] M. H. Yap, M. Goyal, F. M. Osman, R. Martí, E. Denton, A. Juette, and R. Zwiggelaar, “Breast ultrasound lesions recognition: end-to-end deep learning approaches,” Journal of Medical Imaging, vol. 6, no. 1, p. 011007, 2018.
  • [11] N. Cristianini, J. Shawe-Taylor et al., An introduction to support vector machines and other kernel-based learning methods.   Cambridge university press, 2000.
  • [12] E. R. DeLong, D. M. DeLong, and D. L. Clarke-Pearson, “Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach.” Biometrics, vol. 44, no. 3, pp. 837–845, 1988.
  • [13] X. Robin, N. Turck, A. Hainard, N. Tiberti, F. Lisacek, J. Sanchez, and M. Muller, “proc: an open-source package for r and s plus to analyze and compare roc curves,” Bmc Bioinformatics. Mar, vol. 17, p. 12, 2011.
  • [14]

    N. Antropova, B. Q. Huynh, and M. L. Giger, “A deep feature fusion methodology for breast cancer diagnosis demonstrated on three imaging modality datasets,”

    Medical physics, vol. 44, no. 10, pp. 5162–5171, 2017.
  • [15] M. Byra, H. Piotrzkowska-Wróblewska, K. Dobruch-Sobczak, and A. Nowicki, “Combining nakagami imaging and convolutional neural network for breast lesion classification,” in 2017 IEEE International Ultrasonics Symposium (IUS).   IEEE, 2017, pp. 1–4.
  • [16] M. Byra, “Discriminant analysis of neural style representations for breast lesion classification in ultrasound,” Biocybernetics and Biomedical Engineering, vol. 38, no. 3, pp. 684–690, 2018.
  • [17]

    M. Byra, M. Galperin, H. Ojeda-Fournier, L. Olson, M. O’Boyle, C. Comstock, and M. Andre, “Breast mass classification in sonography with transfer learning using a deep convolutional neural network and color conversion,”

    Medical physics, vol. 46, no. 2, pp. 746–755, 2019.
  • [18] S. Han, H.-K. Kang, J.-Y. Jeong, M.-H. Park, W. Kim, W.-C. Bang, and Y.-K. Seong, “A deep learning framework for supporting the classification of breast lesions in ultrasound images,” Physics in Medicine & Biology, vol. 62, no. 19, p. 7714, 2017.
  • [19] X. Qi, L. Zhang, Y. Chen, Y. Pi, Y. Chen, Q. Lv, and Z. Yi, “Automated diagnosis of breast ultrasonography images using deep neural networks,” Medical image analysis, vol. 52, pp. 185–198, 2019.
  • [20]

    D. Cremers, S. J. Osher, and S. Soatto, “Kernel density estimation and intrinsic alignment for knowledge-driven segmentation: Teaching level sets to walk,” in

    Joint Pattern Recognition Symposium.   Springer, 2004, pp. 36–44.