  With Artificial Intelligence (AI) increasingly permeating various aspects of
society, including healthcare, the adoption of the Transformers neural network
architecture is rapidly changing many applications. Transformer is a type of
deep learning architecture initially developed to solve general-purpose Natural
Language Processing (NLP) tasks and has subsequently been adapted in many
fields, including healthcare. In this survey paper, we provide an overview of
how this architecture has been adopted to analyze various forms of data,
including medical imaging, structured and unstructured Electronic Health
Records (EHR), social media, physiological signals, and biomolecular sequences.
Those models could help in clinical diagnosis, report generation, data
reconstruction, and drug/protein synthesis. We identified relevant studies
using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses
(PRISMA) guidelines. We also discuss the benefits and limitations of using
transformers in healthcare and examine issues such as computational cost, model
interpretability, fairness, alignment with human values, ethical implications,
and environmental impact.
