  The increasing popularity of attention mechanisms in deep learning algorithms
for computer vision and natural language processing made these models
attractive to other research domains. In healthcare, there is a strong need for
tools that may improve the routines of the clinicians and the patients.
Naturally, the use of attention-based algorithms for medical applications
occurred smoothly. However, being healthcare a domain that depends on
high-stake decisions, the scientific community must ponder if these
high-performing algorithms fit the needs of medical applications. With this
motto, this paper extensively reviews the use of attention mechanisms in
machine learning (including Transformers) for several medical applications.
This work distinguishes itself from its predecessors by proposing a critical
analysis of the claims and potentialities of attention mechanisms presented in
the literature through an experimental case study on medical image
classification with three different use cases. These experiments focus on the
integrating process of attention mechanisms into established deep learning
architectures, the analysis of their predictive power, and a visual assessment
of their saliency maps generated by post-hoc explanation methods. This paper
concludes with a critical analysis of the claims and potentialities presented
in the literature about attention mechanisms and proposes future research lines
in medical applications that may benefit from these frameworks.
