  Current machine learning methods for medical image analysis primarily focus
on developing models tailored for their specific tasks, utilizing data within
their target domain. These specialized models tend to be data-hungry and often
exhibit limitations in generalizing to out-of-distribution samples. In this
work, we show that employing models that incorporate multiple domains instead
of specialized ones significantly alleviates the limitations observed in
specialized models. We refer to this approach as multi-domain model and compare
its performance to that of specialized models. For this, we introduce the
incorporation of diverse medical image domains, including different imaging
modalities like X-ray, MRI, CT, and ultrasound images, as well as various
viewpoints such as axial, coronal, and sagittal views. Our findings underscore
the superior generalization capabilities of multi-domain models, particularly
in scenarios characterized by limited data availability and
out-of-distribution, frequently encountered in healthcare applications. The
integration of diverse data allows multi-domain models to utilize information
across domains, enhancing the overall outcomes substantially. To illustrate,
for organ recognition, multi-domain model can enhance accuracy by up to 8%
compared to conventional specialized models.
