Monitoring Achilles tendon healing progress in ultrasound imaging with convolutional neural networks

09/11/2019 ∙ by Piotr Woznicki, et al. ∙ 0

Achilles tendon rupture is a debilitating injury, which is typically treated with surgical repair and long-term rehabilitation. The recovery, however, is protracted and often incomplete. Diagnosis, as well as healing progress assessment, are largely based on ultrasound and magnetic resonance imaging. In this paper, we propose an automatic method based on deep learning for analysis of Achilles tendon condition and estimation of its healing progress on ultrasound images. We develop custom convolutional neural networks for classification and regression on healing score and feature extraction. Our models are trained and validated on an acquired dataset of over 250.000 sagittal and over 450.000 axial ultrasound slices. The obtained estimates show a high correlation with the assessment of expert radiologists, with respect to all key parameters describing healing progress. We also observe that parameters associated with i.a. intratendinous healing processes are better modeled with sagittal slices. We prove that ultrasound imaging is quantitatively useful for clinical assessment of Achilles tendon healing process and should be viewed as complementary to magnetic resonance imaging.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 3

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

The Achilles tendon is the largest and strongest tendon in the human body. However, it is one of the most frequently injured tendons, especially among middle-aged people who participate in recreational sports. The incidence of Achilles tendon ruptures has been increasing over the last years [8]. Usually, the diagnosis of an acute rupture is based on detailed musculoskeletal examinations and comprehensive medical history. Ultrasonography (US) and Magnetic Resonance Imaging (MRI) are routinely used for confirming the clinical diagnosis.

The surgical treatment of acute Achilles tendon rupture has been shown to reduce the risk of re-rupture, but it might also lead to a higher complication rate [8]. Furthermore, recent studies show that early functional rehabilitation could also stimulate tendon healing. For the above reasons, regular evaluation of the early tendon healing process is needed to establish patient prognosis and plan further treatment. The US findings correlate with several healing parameters, including cross-sectional area, tendon length or intratendinous morphology and are considered a safe and convenient method of assessing the healing progress [3]. However, some studies have found only a moderate correlation of US findings with clinical assessment of Achilles tendinopathy and clinical outcomes [5].

Quantitative methods based on deep learning are well-suited for modelling the complex relationships between medical images and their interpretation. Recently, approaches using convolutional neural networks (CNNs) have outperformed traditional image analysis methods and proved their usefulness in the analysis of the Achilles tendon MRI scans [4].

In this study, we present a method for the automatic evaluation of the healing process of reconstructed Achilles tendon based on CNNs. We extend the approach proposed in [4] to US images in the axial and the sagittal plane and develop a novel method for healing phase estimation. To our knowledge, there are no other approaches in the literature to quantitatively asses the process of tendon healing through automated analyses of MRI and US imaging. Within this paper we also show that the method applied to MRI cannot by directly transferred to US data, which might result from problematic interpretation of the US images.

More precisely, we first train and evaluate neural networks for the task of binary classification of a single ultrasound slice as healthy or injured. We then present our approaches to modelling the healing progress with respect to 6 key healing parameters. We analyse the applicability of the method using outputs of a pre-trained network with a linear classifier on the PCA-reduced space of the features to assess the progress with the US data. We find that this method fails to learn the accurate representation of the healing phase, therefore we propose an end-to-end CNN performing regression on healing parameters as a new, alternative approach. We further discuss the meaningfulness of the results for US and compare them with MRI results, to finally determine the clinical usefulness of used modalities and applicability of automatic methods for healing assessment.

2 Methods

In this section we describe our method based on the Convolutional Neural Networks. CNNs are discriminative deep architectures, able to extract high-level spatial and configuration information from an image, thus making them suitable for classification of 2D US imaging.

We use models with weights pretrained on ImageNet and train them to explicitly model radiologist assessments. To this end, we modify the architecture of the top dense layer of the CNN in such a way that the output layer performs linear regression on the high-level features from the penultimate layer. For initial tests we use three models of various complexity to eventually select Inception-v3 

[7] architecture as a base for our final solution. These experiments are described as the supervised approach. We then exploit the latent representation and reduce the dimensionality, which makes it possible to obtain a single-number summary of the tendon condition on one US examination. We refer to it as semi-supervised approach. In general, our approach leverages the ability of neural networks to approximate non-linear mappings directly and implicitly accounts for the intermediate feature representations. It maps the images to the tendon healing scores for the different protocols and clinical parameters. We train separate models for both US planes and for all of the ground-truth parameters described in the next subsection.

Figure 1: Achilles healing process for a chosen patient on US imaging. On the sagittal images one can observe the gradual recovery of the fibrillary pattern of tendon fibres with hyperechoic bands. In the axial plane the typical change is the widening of the Achilles tendon and the loss of the hypoechoic fluid collection surrounding the tendon. The images also exemplify certain artifacts typical for ultrasound, including reverberation, refraction and acoustic shadowing.

2.1 Healing progress scoring

Our ground-truth is a survey that has been devised by expert radiologists, in order to quantitatively characterize their subjective assessment of Achilles tendon healing progress based on MRI and US. The survey evaluates the anatomy, metabolic activity and general functionality of the tendon. The following 6 parameters describing the tendon healing process were proposed [4]:

  1. Structural changes within the tendon (SCT)

  2. Tendon thickening (TT)

  3. Sharpness of the tendon edges (STE)

  4. Tendon edema (TE)

  5. Tendon uniformity (TU)

  6. Tissue edema (TisE)

Each parameter is evaluated on a 7-point scale, where 1 corresponds to healthy and 7 to severely injured tendon. We use the scores as ground-truth labels in the training process. Our image dataset is presented in the next subsection.

2.2 Dataset

The original ultrasound dataset includes 49 patients with acute Achilles tendon rupture, all of whom underwent repair surgery and were closely monitored thereafter. The age of patients ranged from 18 to 50 years with a mean age of 36 years. The ultrasound examination was performed at 10 respective intervals: preoperatively, 1 week, 3, 6, 9, 12 weeks after, 4.5, 6, 9 and 12 months after the reconstruction. Additionally, 18 healthy volunteers have been scanned once. For all the examinations a GE 3D high-resolution Voluson E8 Expert ultrasound machine has been used with linear probes 5–18 MHz. The total dataset consists of 565 3D US exams but in this work, we focus on 2D scans only. Clinically, sagittal and axial scanning planes are used interchangeably by rotating the transducer, so we conduct the experiments separately for both. Considering the 2D slices, the final dataset includes 253,639 sagittal scans, 245,366 from patients with ruptured tendon and 8,273 from healthy patients. Alternatively, it consists of 467,548 axial scans, 450,816 injured and 16,732 healthy. The healing progression for an exemplary patient is shown in Fig. 1. Though a detailed analysis can be done only by a trained medical professional, one can observe that the filamentous structures are more visible on the sagittal cross-sections while axial slices present in more details the tissue surrounding, edema and internal tendon pattern.

sagittal axial
Architecture Accuracy Precision Recall Accuracy Precision Recall
AlexNet 0.84600.087 0.9200.08 0.7800.11 0.84300.075 0.9300.06 0.7300.11
Inception-v3 0.91600.049 0.9700.04 0.9000.06 0.90100.052 0.9500.3 0.8700.07
ResNet50 0.90700.039 0.9600.05 0.8900.08 0.91200.046 0.9500.04 0.8800.06
Table 1: Five-fold cross-validation results for the balanced dataset of 2D US scans.

3 Experiments

3.1 Binary classification

We train three network architectures: AlexNet [6], Inception-v3 and ResNet50 [2]

independently on sagittal and axial slices for the task of binary classification of the tendon on a 2D US scan as healthy or injured. The injured class is represented by all the exams of ruptured Achilles tendon performed preoperatively or 1 week after surgery. In order to balance the two classes we use mirroring on the healthy slices and we subsample injured patients for every training epoch.

The accuracy is assessed in 5-fold cross-validation (Tab. 1). ROC and Precision-Recall Curves of the best performing model in terms of highest accuracy (Inception-v3 on sagittal slices) are presented in Fig. 2. For both Inception-v3 and ResNet50 we obtained an accuracy of over on both sagittal and axial scans, which proves that a CNN can be successfully trained on ultrasound data to differentiate between healthy and injured state.

Figure 2: ROC and Precision-Recall Curves for the Inception-v3 on sagittal US images.

We also experiment with the region of interest (ROI) segmentation as a preprocessing step for sagittal scans, applying Active Contours Without Edges [1], which is widely used in the medical field. We hypothesize that focusing exclusively on the tendon region might reduce the noise and artifacts inherently present in US imaging. However, the experiments show lower accuracy with ROI segmentation cropping as compared to non-cropped images, which suggests that the tissues surrounding the Achilles tendon contribute relevant information to the classification.

3.2 Healing progress estimation:

3.2.1 Semi-supervised approach

The neural networks trained for binary classification are used as feature extractors for the task of computing the healing progress score. Principal Component Analysis (PCA) is applied on the feature space to reduce its dimensionality and the first principal component is considered as a representative score for the 2D US scan. For every examination, the aggregate score is calculated as a truncated mean of all 2D scan scores within a single study.

sagittal
Network SCT TT STE TE TU TisE
AlexNet MAE 0.960.41 0.800.27 0.820.29 0.950.41 0.870.39 1.080.47
MAX-AE 1.75 1.32 1.87 1.35 1.61 2.03
Corr 0.530.47 0.690.38 0.110.32 0.680.44 0.310.51 0.220.53
Inception-v3 MAE 0.880.35 0.670.23 0.800.31 0.820.23 0.840.32 0.930.29
MAX-AE 1.69 1.32 1.69 1.31 1.58 1.64
Corr 0.830.44 0.710.40 0.190.34 0.640.47 0.560.40 0.710.40
ResNet50 MAE 0.890.12 0.740.15 0.830.22 0.810.31 0.920.31 0.990.32
MAX-AE 1.53 1.22 1.64 1.43 1.67 1.71
Corr 0.620.31 0.380.51 0.230.41 0.620.51 0.120.43 0.430.50
axial
SCT TT STE TE TU TisE
AlexNet MAE 0.980.39 0.830.33 0.820.35 0.940.52 0.950.50 0.860.28
MAX-AE 1.79 1.36 1.86 1.41 1.61 1.59
Corr 0.450.33 0.620.39 0.200.45 0.600.47 0.030.42 0.590.40
Inception-v3 MAE 1.030.46 0.700.24 0.760.26 0.860.19 0.870.32 0.850.25
MAX-AE 2.52 1.45 1.45 1.24 1.67 1.56
Corr 0.770.47 0.690.40 0.220.37 0.650.41 0.550.44 0.720.41
ResNet50 MAE 1.050.31 0.780.33 0.800.24 1.020.25 0.870.15 0.910.28
MAX-AE 1.98 1.51 1.59 1.63 1.45 1.57
Corr 0.520.41 0.470.44 0.220.35 0.650.55 0.180.54 0.570.39
Table 2: 5CV results for the tendon healing progress using end-to-end approach

Although this method was proven to work for MRI scans [4]

, for ultrasound we observed a very weak correlation with actual healing parameters, which should be attributed to lower variance preserved by the first principal components and higher variance between scans from one examination. Therefore we do not present the results here. We believe that speckle noise, a random granular pattern produced mainly by multiplicative disturbances, as well as frequent artifacts are the main reasons for the weak performance of the tested method.

3.2.2 Supervised approach

Healing scores are evaluated in 5-fold cross-validation using mean absolute error (MAE), maximal absolute error for a single exam (MAX-AE) and mean correlation, computed with the use of Fisher Z-Transformation (Tab.

2).

We observe a good correspondence between the estimated healing scores and the experts’ assessment, with MAE ranging from to , on a 7 point scale. For all the networks we notice a positive mean correlation of our method’s output and healing parameters. Although the results are consistent between different networks, Inception-v3 usually achieves the best fit and the simplest network architecture, AlexNet, performs noticeably worse. Two healing parameters, SCT and TT are more accurately estimated on sagittal rather than axial US images and one parameter, TisE, vice versa.

The final evaluation of the regression task has been done on a separate test set, consisting of 4 injured patients who underwent a full rehabilitation process, i.e. 40 studies in total (Tab. 3). For the best performing Inception-v3, we report MAE ranging from to and correlations in the range of to . The resulting healing progress for a selected parameter is compared with radiologist evaluation in Fig. 3. In general, axial and sagittal models give similar results, which tend to correlate well with ground-truth labels.

sagittal
Network SCT TT STE TE TU TisE
AlexNet MAE 0.900.31 0.630.12 0.690.31 0.810.11 0.890.20 1.010.35
Corr 0.550.15 0.700.24 0.220.49 0.610.28 0.120.43 0.280.27
Inception-v3 MAE 0.810.38 0.630.06 0.560.18 0.850.20 0.540.04 0.870.29
Corr 0.800.39 0.770.28 0.310.33 0.520.36 0.690.34 0.620.52
ResNet50 MAE 0.880.33 0.650.15 0.660.09 0.830.25 0.750.12 0.930.22
Corr 0.600.32 0.550.38 0.250.27 0.550.41 0.340.29 0.560.38
axial
SCT TT STE TE TU TisE
AlexNet MAE 1.120.36 0.810.29 0.580.12 0.870.19 0.700.24 0.850.25
Corr 0.460.50 0.540.32 0.260.41 0.380.38 0.120.34 0.700.31
Inception-v3 MAE 0.840.54 0.751.45 0.580.10 0.830.10 0.530.16 0.830.30
Corr 0.690.49 0.680.41 0.450.15 0.510.42 0.660.16 0.680.39
ResNet50 MAE 0.920.37 0.760.32 0.680.08 0.810.17 0.650.20 0.940.11
Corr 0.550.41 0.570.38 0.350.31 0.440.39 0.390.35 0.610.33
Table 3: Results for the tendon healing progress using end-to-end approach on the test dataset

4 Discussion

We show that a neural network learns to extract features from the US images which strongly correlate with the healing progress score assigned by expert radiologists. Out of the three healing parameters: tendon uniformity (TU), structural changes (SCT) and tendon thickening (TT), which correspond to morphological changes within the Achilles tendon and are typically evaluated in the longitudinal axis, SCT and TT are better modeled by the sagittal ultrasound, while TU still retains MAE of point. On the other hand, sharpness of the tendon edges (STE), tendon edema (TE) and tissue edema (TisE) are typically evaluated on axial slices and for STE and TisE, all our networks achieve lower MAE and higher mean correlation when trained in the axial plane.

In comparison with the results from [4], we notice that a convolutional neural network is able to achieve a better accuracy of binary classification on MRI data rather than US data (99.83% vs. 91.6% for the best respective models). Furthermore, a high correlation of automated method output with the ground truth in terms of three parameters: TE, TisE and STE has been reported for MRI scans. MR-acquired stacks of axial images of the Achilles tendon have a major limitation in the form of lower spatial resolution along the longitudinal axis, which is determined by the slice selection pulse. Because of this spatial anisotropy, they are not suitable for assessing healing parameters, which rely on the intratendinous processes or the alignment of fibrous bands.

Figure 3: Inception-v3 results for the TU parameter on test dataset (correlations refer to sagittal scans).

The results suggest that features extracted by deep learning models from MR and US imaging focus on different qualities of the rehabilitation process. This indicates that ultrasound should be viewed as an imaging method that complements MRI rather than one that competes with MRI in the evaluation of musculoskeletal abnormalities. It should be noted, however, that the previous work on MRI was validated on a smaller dataset and did not apply the supervised end-to-end approach, which limits us to an indirect qualitative comparison.

5 Conclusions

In this paper, we proposed deep learning models that achieve high performance in clinical classification and healing phase estimation of ruptured Achilles tendon. We have compared two approaches to modelling tendon rehabilitation progress and shown that the supervised method is superior to the semi-supervised method. Currently, monitoring the healing process requires a radiologist to analyze US and MRI data and subjectively evaluate the condition of the tendon.

As suggested in [3], tendon morphology may be the more robust measure to gauge patient healing progress over time compared to mechanical properties of the tendon. Therefore, we believe that a model which accurately estimates healing parameters from standardized images may be useful in clinical practice.

Future studies are needed to improve the generalizability of deep learning models for medical imaging in musculoskeletal disorders and to determine the effect of model assistance in the clinical setting.

References

  • [1] T. F. Chan and L. A. Vese (2001-02) Active contours without edges. Trans. Img. Proc. 10 (2), pp. 266–277. External Links: ISSN 1057-7149, Link, Document Cited by: §3.1.
  • [2] K. He, X. Zhang, S. Ren, and J. Sun (2015) Deep residual learning for image recognition. CoRR abs/1512.03385. External Links: Link, 1512.03385 Cited by: §3.1.
  • [3] K. Hiramatsu, A. Tsujii, N. Nakamura, and T. Mitsuoka (2018-08) Ultrasonographic Evaluation of the Early Healing Process After Achilles Tendon Repair. Orthop J Sports Med 6 (8), pp. 2325967118789883. Cited by: §1, §5.
  • [4] N. Kapinski, J. Zielinski, B. A. Borucki, T. Trzcinski, B. Ciszkowska-Lyson, and K. S. Nowinski (2018) Estimating achilles tendon healing progress with convolutional neural networks. In Medical Image Computing and Computer Assisted Intervention – MICCAI 2018, A. F. Frangi, J. A. Schnabel, C. Davatzikos, C. Alberola-López, and G. Fichtinger (Eds.), Cham, pp. 949–957. External Links: ISBN 978-3-030-00934-2 Cited by: §1, §1, §2.1, §3.2.1, §4.
  • [5] K. M. Khan, B. B. Forster, J. Robinson, Y. Cheong, L. Louis, L. Maclean, and J. E. Taunton (2003) Are ultrasound and magnetic resonance imaging of value in assessment of achilles tendon disorders? a two year prospective study. British Journal of Sports Medicine 37 (2), pp. 149–153. External Links: Document, ISSN 0306-3674, Link, https://bjsm.bmj.com/content/37/2/149.full.pdf Cited by: §1.
  • [6] A. Krizhevsky, I. Sutskever, and G. E. Hinton (2012) ImageNet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems 25, F. Pereira, C. J. C. Burges, L. Bottou, and K. Q. Weinberger (Eds.), pp. 1097–1105. External Links: Link Cited by: §3.1.
  • [7] C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna (2015)

    Rethinking the inception architecture for computer vision

    .
    CoRR abs/1512.00567. External Links: Link, 1512.00567 Cited by: §2.
  • [8] K. Zhou, L. Song, P. Zhang, C. Wang, and W. Wang (2018) Surgical versus non-surgical methods for acute achilles tendon rupture: a meta-analysis of randomized controlled trials. The Journal of Foot and Ankle Surgery 57 (6), pp. 1191 – 1199. External Links: ISSN 1067-2516, Document, Link Cited by: §1, §1.