  Recent advancements in areas such as natural language processing and computer
vision rely on intricate and massive models that have been trained using vast
amounts of unlabelled or partly labeled data and training or deploying these
state-of-the-art methods to resource constraint environments has been a
challenge. Galaxy morphologies are crucial to understanding the processes by
which galaxies form and evolve. Efficient methods to classify galaxy
morphologies are required to extract physical information from modern-day
astronomy surveys. In this paper, we introduce Astroformer, a method to learn
from less amount of data. We propose using a hybrid transformer-convolutional
architecture drawing much inspiration from the success of CoAtNet and MaxViT.
Concretely, we use the transformer-convolutional hybrid with a new stack design
for the network, a different way of creating a relative self-attention layer,
and pair it with a careful selection of data augmentation and regularization
techniques. Our approach sets a new state-of-the-art on predicting galaxy
morphologies from images on the Galaxy10 DECals dataset, a science objective,
which consists of 17736 labeled images achieving 94.86% top-$1$ accuracy,
beating the current state-of-the-art for this task by 4.62%. Furthermore, this
approach also sets a new state-of-the-art on CIFAR-100 and Tiny ImageNet. We
also find that models and training methods used for larger datasets would often
not work very well in the low-data regime.
