  Spatio-temporal modeling is foundational for smart city applications, yet it
is often hindered by data scarcity in many cities and regions. To bridge this
gap, we propose a novel generative pre-training framework, GPD, for
spatio-temporal few-shot learning with urban knowledge transfer. Unlike
conventional approaches that heavily rely on common feature extraction or
intricate few-shot learning designs, our solution takes a novel approach by
performing generative pre-training on a collection of neural network parameters
optimized with data from source cities. We recast spatio-temporal few-shot
learning as pre-training a generative diffusion model, which generates tailored
neural networks guided by prompts, allowing for adaptability to diverse data
distributions and city-specific characteristics. GPD employs a
Transformer-based denoising diffusion model, which is model-agnostic to
integrate with powerful spatio-temporal neural networks. By addressing
challenges arising from data gaps and the complexity of generalizing knowledge
across cities, our framework consistently outperforms state-of-the-art
baselines on multiple real-world datasets for tasks such as traffic speed
prediction and crowd flow prediction. The implementation of our approach is
available: https://github.com/tsinghua-fib-lab/GPD.
