  Motivation: Medical image analysis involves tasks to assist physicians in
qualitative and quantitative analysis of lesions or anatomical structures,
significantly improving the accuracy and reliability of diagnosis and
prognosis. Traditionally, these tasks are finished by physicians or medical
physicists and lead to two major problems: (i) low efficiency; (ii) biased by
personal experience. In the past decade, many machine learning methods have
been applied to accelerate and automate the image analysis process. Compared to
the enormous deployments of supervised and unsupervised learning models,
attempts to use reinforcement learning in medical image analysis are scarce.
This review article could serve as the stepping-stone for related research.
Significance: From our observation, though reinforcement learning has gradually
gained momentum in recent years, many researchers in the medical analysis field
find it hard to understand and deploy in clinics. One cause is lacking
well-organized review articles targeting readers lacking professional computer
science backgrounds. Rather than providing a comprehensive list of all
reinforcement learning models in medical image analysis, this paper may help
the readers to learn how to formulate and solve their medical image analysis
research as reinforcement learning problems. Approach & Results: We selected
published articles from Google Scholar and PubMed. Considering the scarcity of
related articles, we also included some outstanding newest preprints. The
papers are carefully reviewed and categorized according to the type of image
analysis task. We first review the basic concepts and popular models of
reinforcement learning. Then we explore the applications of reinforcement
learning models in landmark detection. Finally, we conclude the article by
discussing the reviewed reinforcement learning approaches' limitations and
possible improvements.
