  This paper demonstrates that pre-trained language models (PLMs) are strong
foundation models for on-device meteorological variables modeling. We present
LM-Weather, a generic approach to taming PLMs, that have learned massive
sequential knowledge from the universe of natural language databases, to
acquire an immediate capability to obtain highly customized models for
heterogeneous meteorological data on devices while keeping high efficiency.
Concretely, we introduce a lightweight personalized adapter into PLMs and
endows it with weather pattern awareness. During communication between clients
and the server, low-rank-based transmission is performed to effectively fuse
the global knowledge among devices while maintaining high communication
efficiency and ensuring privacy. Experiments on real-wold dataset show that
LM-Weather outperforms the state-of-the-art results by a large margin across
various tasks (e.g., forecasting and imputation at different scales). We
provide extensive and in-depth analyses experiments, which verify that
LM-Weather can (1) indeed leverage sequential knowledge from natural language
to accurately handle meteorological sequence, (2) allows each devices obtain
highly customized models under significant heterogeneity, and (3) generalize
under data-limited and out-of-distribution (OOD) scenarios.
