  Recent advances in deep learning and computer vision have reduced many
barriers to automated medical image analysis, allowing algorithms to process
label-free images and improve performance. However, existing techniques have
extreme computational requirements and drop a lot of performance with a
reduction in batch size or training epochs. This paper presents Cross
Architectural - Self Supervision (CASS), a novel self-supervised learning
approach that leverages Transformer and CNN simultaneously. Compared to the
existing state of the art self-supervised learning approaches, we empirically
show that CASS-trained CNNs and Transformers across four diverse datasets
gained an average of 3.8% with 1% labeled data, 5.9% with 10% labeled data, and
10.13% with 100% labeled data while taking 69% less time. We also show that
CASS is much more robust to changes in batch size and training epochs. Notably,
one of the test datasets comprised histopathology slides of an autoimmune
disease, a condition with minimal data that has been underrepresented in
medical imaging. The code is open source and is available on GitHub.
