  In this paper, we introduce PredBench, a benchmark tailored for the holistic
evaluation of spatio-temporal prediction networks. Despite significant progress
in this field, there remains a lack of a standardized framework for a detailed
and comparative analysis of various prediction network architectures. PredBench
addresses this gap by conducting large-scale experiments, upholding
standardized and appropriate experimental settings, and implementing
multi-dimensional evaluations. This benchmark integrates 12 widely adopted
methods with 15 diverse datasets across multiple application domains, offering
extensive evaluation of contemporary spatio-temporal prediction networks.
Through meticulous calibration of prediction settings across various
applications, PredBench ensures evaluations relevant to their intended use and
enables fair comparisons. Moreover, its multi-dimensional evaluation framework
broadens the analysis with a comprehensive set of metrics, providing deep
insights into the capabilities of models. The findings from our research offer
strategic directions for future developments in the field. Our codebase is
available at https://github.com/OpenEarthLab/PredBench.
