  As the potential of foundation models in visual tasks has garnered
significant attention, pretraining these models before downstream tasks has
become a crucial step. The three key factors in pretraining foundation models
are the pretraining method, the size of the pretraining dataset, and the number
of model parameters. Recently, research in the remote sensing field has focused
primarily on the pretraining method and the size of the dataset, with limited
emphasis on the number of model parameters. This paper addresses this gap by
examining the effect of increasing the number of model parameters on the
performance of foundation models in downstream tasks such as rotated object
detection and semantic segmentation. We pretrained foundation models with
varying numbers of parameters, including 86M, 605.26M, 1.3B, and 2.4B, to
determine whether performance in downstream tasks improved with an increase in
parameters. To the best of our knowledge, this is the first billion-scale
foundation model in the remote sensing field. Furthermore, we propose an
effective method for scaling up and fine-tuning a vision transformer in the
remote sensing field. To evaluate general performance in downstream tasks, we
employed the DOTA v2.0 and DIOR-R benchmark datasets for rotated object
detection, and the Potsdam and LoveDA datasets for semantic segmentation.
Experimental results demonstrated that, across all benchmark datasets and
downstream tasks, the performance of the foundation models and data efficiency
improved as the number of parameters increased. Moreover, our models achieve
the state-of-the-art performance on several datasets including DIOR-R, Postdam,
and LoveDA.
