  Understanding 3D scenes is a critical prerequisite for autonomous agents.
Recently, LiDAR and other sensors have made large amounts of data available in
the form of temporal sequences of point cloud frames. In this work, we propose
a novel problem -- sequential scene flow estimation (SSFE) -- that aims to
predict 3D scene flow for all pairs of point clouds in a given sequence. This
is unlike the previously studied problem of scene flow estimation which focuses
on two frames.
  We introduce the SPCM-Net architecture, which solves this problem by
computing multi-scale spatiotemporal correlations between neighboring point
clouds and then aggregating the correlation across time with an order-invariant
recurrent unit. Our experimental evaluation confirms that recurrent processing
of point cloud sequences results in significantly better SSFE compared to using
only two frames. Additionally, we demonstrate that this approach can be
effectively modified for sequential point cloud forecasting (SPF), a related
problem that demands forecasting future point cloud frames.
  Our experimental results are evaluated using a new benchmark for both SSFE
and SPF consisting of synthetic and real datasets. Previously, datasets for
scene flow estimation have been limited to two frames. We provide non-trivial
extensions to these datasets for multi-frame estimation and prediction. Due to
the difficulty of obtaining ground truth motion for real-world datasets, we use
self-supervised training and evaluation metrics. We believe that this benchmark
will be pivotal to future research in this area. All code for benchmark and
models will be made accessible.
