Vision Transformers for femur fracture classification

by   Leonardo Tanzi, et al.

Objectives: In recent years, the scientific community has focused on the development of Computer-Aided Diagnosis (CAD) tools that could improve bone fractures' classification. However, the results of the classification of fractures in subtypes with the proposed datasets were far from optimal. This paper proposes a very recent and outperforming deep learning technique, the Vision Transformer (ViT), in order to improve the fracture classification, by exploiting its self-attention mechanism. Methods: 4207 manually annotated images were used and distributed, by following the AO/OTA classification, in different fracture types, the largest labeled dataset of proximal femur fractures used in literature. The ViT architecture was used and compared with a classic Convolutional Neural Network (CNN) and a multistage architecture composed by successive CNNs in cascade. To demonstrate the reliability of this approach, 1) the attention maps were used to visualize the most relevant areas of the images, 2) the performance of a generic CNN and ViT was also compared through unsupervised learning techniques, and 3) 11 specialists were asked to evaluate and classify 150 proximal femur fractures' images with and without the help of the ViT. Results: The ViT was able to correctly predict 83 Precision, recall and F1-score were 0.77 (CI 0.64-0.90), 0.76 (CI 0.62-0.91) and 0.77 (CI 0.64-0.89), respectively. The average specialists' diagnostic improvement was 29 Conclusions: This paper showed the potential of Transformers in bone fracture classification. For the first time, good results were obtained in sub-fractures with the largest and richest dataset ever.


page 4

page 7

page 8

page 9


ConvFormer: Closing the Gap Between CNN and Vision Transformers

Vision transformers have shown excellent performance in computer vision ...

Towards an Interactive and Interpretable CAD System to Support Proximal Femur Fracture Classification

Fractures of the proximal femur represent a critical entity in the weste...

Vision Transformers For Weeds and Crops Classification Of High Resolution UAV Images

Crop and weed monitoring is an important challenge for agriculture and f...

Locality Guidance for Improving Vision Transformers on Tiny Datasets

While the Vision Transformer (VT) architecture is becoming trendy in com...

Seeking an Optimal Approach for Computer-Aided Pulmonary Embolism Detection

Pulmonary embolism (PE) represents a thrombus ("blood clot"), usually or...

Combining EfficientNet and Vision Transformers for Video Deepfake Detection

Deepfakes are the result of digital manipulation to obtain credible vide...

A Survey and Approach to Chart Classification

Charts represent an essential source of visual information in documents ...

Please sign up or login with your details

Forgot password? Click here to reset