  Previous works on human motion prediction follow the pattern of building a
mapping relation between the sequence observed and the one to be predicted.
However, due to the inherent complexity of multivariate time series data, it
still remains a challenge to find the extrapolation relation between motion
sequences. In this paper, we present a new prediction pattern, which introduces
previously overlooked human poses, to implement the prediction task from the
view of interpolation. These poses exist after the predicted sequence, and form
the privileged sequence. To be specific, we first propose an InTerPolation
learning Network (ITP-Network) that encodes both the observed sequence and the
privileged sequence to interpolate the in-between predicted sequence, wherein
the embedded Privileged-sequence-Encoder (Priv-Encoder) learns the privileged
knowledge (PK) simultaneously. Then, we propose a Final Prediction Network
(FP-Network) for which the privileged sequence is not observable, but is
equipped with a novel PK-Simulator that distills PK learned from the previous
network. This simulator takes as input the observed sequence, but approximates
the behavior of Priv-Encoder, enabling FP-Network to imitate the interpolation
process. Extensive experimental results demonstrate that our prediction pattern
achieves state-of-the-art performance on benchmarked H3.6M, CMU-Mocap and 3DPW
datasets in both short-term and long-term predictions.
