  We propose a novel method of finding principal components in multivariate
data sets that lie on an embedded nonlinear Riemannian manifold within a
higher-dimensional space. Our aim is to extend the geometric interpretation of
PCA, while being able to capture non-geodesic modes of variation in the data.
We introduce the concept of a principal sub-manifold, a manifold passing
through a reference point, and at any point on the manifold extending in the
direction of highest variation in the space spanned by the eigenvectors of the
local tangent space PCA. Compared to recent work for the case where the
sub-manifold is of dimension one Panaretos et al. (2014)$-$essentially a curve
lying on the manifold attempting to capture one-dimensional variation$-$the
current setting is much more general. The principal sub-manifold is therefore
an extension of the principal flow, accommodating to capture higher dimensional
variation in the data. We show the principal sub-manifold yields the ball
spanned by the usual principal components in Euclidean space. By means of
examples, we illustrate how to find, use and interpret a principal sub-manifold
and we present an application in shape analysis.
