  We propose a novel framework to learn the spatiotemporal variability in
longitudinal 3D shape data sets, which contain observations of objects that
evolve and deform over time. This problem is challenging since surfaces come
with arbitrary parameterizations and thus, they need to be spatially
registered. Also, different deforming objects, also called 4D surfaces, evolve
at different speeds and thus they need to be temporally aligned. We solve this
spatiotemporal registration problem using a Riemannian approach. We treat a 3D
surface as a point in a shape space equipped with an elastic Riemannian metric
that measures the amount of bending and stretching that the surfaces undergo. A
4D surface can then be seen as a trajectory in this space. With this
formulation, the statistical analysis of 4D surfaces can be cast as the problem
of analyzing trajectories embedded in a nonlinear Riemannian manifold. However,
performing the spatiotemporal registration, and subsequently computing
statistics, on such nonlinear spaces is not straightforward as they rely on
complex nonlinear optimizations. Our core contribution is the mapping of the
surfaces to the space of Square-Root Normal Fields where the L2 metric is
equivalent to the partial elastic metric in the space of surfaces. Thus, by
solving the spatial registration in the SRNF space, the problem of analyzing 4D
surfaces becomes the problem of analyzing trajectories embedded in the SRNF
space, which has a Euclidean structure. In this paper, we develop the building
blocks that enable such analysis. These include: (1) the spatiotemporal
registration of arbitrarily parameterized 4D surfaces in the presence of large
elastic deformations and large variations in their execution rates; (2) the
computation of geodesics between 4D surfaces; (3) the computation of
statistical summaries; and (4) the synthesis of random 4D surfaces.
