  Multi-modal large language models (MLLMs) have demonstrated remarkable
success in vision and visual-language tasks within the natural image domain.
Owing to the significant diversities between the natural and remote sensing
(RS) images, the development of MLLMs in the RS domain is still in the infant
stage. To fill the gap, a pioneer MLLM named EarthGPT integrating various
multi-sensor RS interpretation tasks uniformly is proposed in this paper for
universal RS image comprehension. In EarthGPT, three key techniques are
developed including a visual-enhanced perception mechanism, a cross-modal
mutual comprehension approach, and a unified instruction tuning method for
multi-sensor multi-task in the RS domain. More importantly, a dataset named
MMRS-1M featuring large-scale multi-sensor multi-modal RS instruction-following
is constructed, comprising over 1M image-text pairs based on 34 existing
diverse RS datasets and including multi-sensor images such as optical,
synthetic aperture radar (SAR), and infrared. The MMRS-1M dataset addresses the
drawback of MLLMs on RS expert knowledge and stimulates the development of
MLLMs in the RS domain. Extensive experiments are conducted, demonstrating the
EarthGPT's superior performance in various RS visual interpretation tasks
compared with the other specialist models and MLLMs, proving the effectiveness
of the proposed EarthGPT and offering a versatile paradigm for open-set
reasoning tasks.
