  Recently, large language models (LLMs) have taken the spotlight in natural
language processing. Further, integrating LLMs with vision enables the users to
explore emergent abilities with multimodal data. Visual language models (VLMs),
such as LLaVA, Flamingo, or CLIP, have demonstrated impressive performance on
various visio-linguistic tasks. Consequently, there are enormous applications
of large models that could be potentially used in the biomedical imaging field.
Along that direction, there is a lack of related work to show the ability of
large models to diagnose the diseases. In this work, we study the zero-shot and
few-shot robustness of VLMs on the medical imaging analysis tasks. Our
comprehensive experiments demonstrate the effectiveness of VLMs in analyzing
biomedical images such as brain MRIs, microscopic images of blood cells, and
chest X-rays.
