  The pandemic resulted in vast repositories of unstructured data, including
radiology reports, due to increased medical examinations. Previous research on
automated diagnosis of COVID-19 primarily focuses on X-ray images, despite
their lower precision compared to computed tomography (CT) scans. In this work,
we leverage unstructured data from a hospital and harness the fine-grained
details offered by CT scans to perform zero-shot multi-label classification
based on contrastive visual language learning. In collaboration with human
experts, we investigate the effectiveness of multiple zero-shot models that aid
radiologists in detecting pulmonary embolisms and identifying intricate lung
details like ground glass opacities and consolidations. Our empirical analysis
provides an overview of the possible solutions to target such fine-grained
tasks, so far overlooked in the medical multimodal pretraining literature. Our
investigation promises future advancements in the medical image analysis
community by addressing some challenges associated with unstructured data and
fine-grained multi-label classification.
