  Galaxy morphology reflects structural properties which contribute to
understand the formation and evolution of galaxies. Deep convolutional networks
have proven to be very successful in learning hidden features that allow for
unprecedented performance on galaxy morphological classification. Such networks
mostly follow the supervised learning paradigm which requires sufficient
labelled data for training. However, it is an expensive and complicated process
of labeling for million galaxies, particularly for the forthcoming survey
projects. In this paper, we present an approach based on contrastive learning
with aim for learning galaxy morphological visual representation using only
unlabeled data. Considering the properties of low semantic information and
contour dominated of galaxy image, the feature extraction layer of the proposed
method incorporates vision transformers and convolutional network to provide
rich semantic representation via the fusion of the multi-hierarchy features. We
train and test our method on 3 classifications of datasets from Galaxy Zoo 2
and SDSS-DR17, and 4 classifications from Galaxy Zoo DECaLS. The testing
accuracy achieves 94.7%, 96.5% and 89.9% respectively. The experiment of cross
validation demonstrates our model possesses transfer and generalization ability
when applied to the new datasets. The code that reveals our proposed method and
pretrained models are publicly available and can be easily adapted to new
surveys.
