  Fast screening and diagnosis are critical in COVID-19 patient treatment. In
addition to the gold standard RT-PCR, radiological imaging like X-ray and CT
also works as an important means in patient screening and follow-up. However,
due to the excessive number of patients, writing reports becomes a heavy burden
for radiologists. To reduce the workload of radiologists, we propose DeltaNet
to generate medical reports automatically. Different from typical image
captioning approaches that generate reports with an encoder and a decoder,
DeltaNet applies a conditional generation process. In particular, given a
medical image, DeltaNet employs three steps to generate a report: 1) first
retrieving related medical reports, i.e., the historical reports from the same
or similar patients; 2) then comparing retrieved images and current image to
find the differences; 3) finally generating a new report to accommodate
identified differences based on the conditional report. We evaluate DeltaNet on
a COVID-19 dataset, where DeltaNet outperforms state-of-the-art approaches.
Besides COVID-19, the proposed DeltaNet can be applied to other diseases as
well. We validate its generalization capabilities on the public IU-Xray and
MIMIC-CXR datasets for chest-related diseases. Code is available at
\url{https://github.com/LX-doctorAI1/DeltaNet}.
