  Automatic radiology report generation can alleviate the workload for
physicians and minimize regional disparities in medical resources, therefore
becoming an important topic in the medical image analysis field. It is a
challenging task, as the computational model needs to mimic physicians to
obtain information from multi-modal input data (i.e., medical images, clinical
information, medical knowledge, etc.), and produce comprehensive and accurate
reports. Recently, numerous works emerged to address this issue using deep
learning-based methods, such as transformers, contrastive learning, and
knowledge-base construction. This survey summarizes the key techniques
developed in the most recent works and proposes a general workflow for deep
learning-based report generation with five main components, including
multi-modality data acquisition, data preparation, feature learning, feature
fusion/interaction, and report generation. The state-of-the-art methods for
each of these components are highlighted. Additionally, training strategies,
public datasets, evaluation methods, current challenges, and future directions
in this field are summarized. We have also conducted a quantitative comparison
between different methods under the same experimental setting. This is the most
up-to-date survey that focuses on multi-modality inputs and data fusion for
radiology report generation. The aim is to provide comprehensive and rich
information for researchers interested in automatic clinical report generation
and medical image analysis, especially when using multimodal inputs, and assist
them in developing new algorithms to advance the field.
