  This paper investigates the generalization of Principal Component Analysis
(PCA) to Riemannian manifolds. We first propose a new and general type of
family of subspaces in manifolds that we call barycentric subspaces. They are
implicitly defined as the locus of points which are weighted means of $k+1$
reference points. As this definition relies on points and not on tangent
vectors, it can also be extended to geodesic spaces which are not Riemannian.
For instance, in stratified spaces, it naturally allows principal subspaces
that span several strata, which is impossible in previous generalizations of
PCA. We show that barycentric subspaces locally define a submanifold of
dimension k which generalizes geodesic subspaces.Second, we rephrase PCA in
Euclidean spaces as an optimization on flags of linear subspaces (a hierarchy
of properly embedded linear subspaces of increasing dimension). We show that
the Euclidean PCA minimizes the Accumulated Unexplained Variances by all the
subspaces of the flag (AUV). Barycentric subspaces are naturally nested,
allowing the construction of hierarchically nested subspaces. Optimizing the
AUV criterion to optimally approximate data points with flags of affine spans
in Riemannian manifolds lead to a particularly appealing generalization of PCA
on manifolds called Barycentric Subspaces Analysis (BSA).
