  Spatio-temporal forecasting, pivotal in numerous fields, hinges on the
delicate equilibrium between isolating nuanced patterns and sifting out noise.
To tackle this, we introduce Sparse Regression-based Vector Quantization (SVQ),
a novel technique that leverages sparse regression for succinct representation,
an approach theoretically and practically favored over classical
clustering-based vector quantization methods. This approach preserves critical
details from the original vectors using a regression model while filtering out
noise via sparse design. Moreover, we approximate the sparse regression process
using a blend of a two-layer MLP and an extensive codebook. This approach not
only substantially cuts down on computational costs but also grants SVQ
differentiability and training simplicity, resulting in a notable enhancement
of performance. Our empirical studies on five spatial-temporal benchmark
datasets demonstrate that SVQ achieves state-of-the-art results. Specifically,
on the WeatherBench-S temperature dataset, SVQ improves the top baseline by
7.9%. In video prediction benchmarks-Human, KTH, and KittiCaltech-it reduces
MAE by an average of 9.4% and improves image quality by 17.3% (LPIPS).
