A survey on attention mechanisms for medical applications: are we moving towards better algorithms?

04/26/2022
by   Tiago Gonçalves, et al.
29

The increasing popularity of attention mechanisms in deep learning algorithms for computer vision and natural language processing made these models attractive to other research domains. In healthcare, there is a strong need for tools that may improve the routines of the clinicians and the patients. Naturally, the use of attention-based algorithms for medical applications occurred smoothly. However, being healthcare a domain that depends on high-stake decisions, the scientific community must ponder if these high-performing algorithms fit the needs of medical applications. With this motto, this paper extensively reviews the use of attention mechanisms in machine learning (including Transformers) for several medical applications. This work distinguishes itself from its predecessors by proposing a critical analysis of the claims and potentialities of attention mechanisms presented in the literature through an experimental case study on medical image classification with three different use cases. These experiments focus on the integrating process of attention mechanisms into established deep learning architectures, the analysis of their predictive power, and a visual assessment of their saliency maps generated by post-hoc explanation methods. This paper concludes with a critical analysis of the claims and potentialities presented in the literature about attention mechanisms and proposes future research lines in medical applications that may benefit from these frameworks.

READ FULL TEXT

page 20

page 21

page 22

page 23

page 24

page 25

research
12/30/2021

Attention mechanisms and deep learning for machine vision: A survey of the state of the art

With the advent of state of the art nature-inspired pure attention based...
research
03/27/2022

A General Survey on Attention Mechanisms in Deep Learning

Attention is an important mechanism that can be employed for a variety o...
research
11/12/2020

Natural Language Processing to Detect Cognitive Concerns in Electronic Health Records Using Deep Learning

Dementia is under-recognized in the community, under-diagnosed by health...
research
11/12/2020

A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges

Uncertainty quantification (UQ) plays a pivotal role in reduction of unc...
research
07/19/2022

Towards Trustworthy Healthcare AI: Attention-Based Feature Learning for COVID-19 Screening With Chest Radiography

Building AI models with trustworthiness is important especially in regul...
research
02/15/2022

The Quarks of Attention

Attention plays a fundamental role in both natural and artificial intell...

Please sign up or login with your details

Forgot password? Click here to reset