  Human motion prediction is challenging due to the complex spatiotemporal
feature modeling. Among all methods, graph convolution networks (GCNs) are
extensively utilized because of their superiority in explicit connection
modeling. Within a GCN, the graph correlation adjacency matrix drives feature
aggregation and is the key to extracting predictive motion features.
State-of-the-art methods decompose the spatiotemporal correlation into spatial
correlations for each frame and temporal correlations for each joint. Directly
parameterizing these correlations introduces redundant parameters to represent
common relations shared by all frames and all joints. Besides, the
spatiotemporal graph adjacency matrix is the same for different motion samples
and cannot reflect sample-wise correspondence variances. To overcome these two
bottlenecks, we propose dynamic spatiotemporal decompose GC (DSTD-GC), which
only takes 28.6% parameters of the state-of-the-art GC. The key of DSTD-GC is
constrained dynamic correlation modeling, which explicitly parameterizes the
common static constraints as a spatial/temporal vanilla adjacency matrix shared
by all frames/joints and dynamically extracts correspondence variances for each
frame/joint with an adjustment modeling function. For each sample, the common
constrained adjacency matrices are fixed to represent generic motion patterns,
while the extracted variances complete the matrices with specific pattern
adjustments. Meanwhile, we mathematically reformulate GCs on spatiotemporal
graphs into a unified form and find that DSTD-GC relaxes certain constraints of
other GC, which contributes to a better representation capability. By combining
DSTD-GC with prior knowledge, we propose a powerful spatiotemporal GCN called
DSTD-GCN, which outperforms SOTA methods by $3.9\% \sim 8.7\%$ in prediction
accuracy with $55.0\% \sim 96.9\%$ fewer parameters.
