  In recent years, the research landscape of machine learning in medical
imaging has changed drastically from supervised to semi-, weakly- or
unsupervised methods. This is mainly due to the fact that ground-truth labels
are time-consuming and expensive to obtain manually. Generating labels from
patient metadata might be feasible but it suffers from user-originated errors
which introduce biases. In this work, we propose an unsupervised approach for
automatically clustering and categorizing large-scale medical image datasets,
with a focus on cardiac MR images, and without using any labels. We
investigated the end-to-end training using both class-balanced and imbalanced
large-scale datasets. Our method was able to create clusters with high purity
and achieved over 0.99 cluster purity on these datasets. The results
demonstrate the potential of the proposed method for categorizing unstructured
large medical databases, such as organizing clinical PACS systems in hospitals.
