  Non-Euclidean constraints are inherent in many kinds of data in computer
vision and machine learning, typically as a result of specific invariance
requirements that need to be respected during high-level inference. Often,
these geometric constraints can be expressed in the language of Riemannian
geometry, where conventional vector space machine learning does not apply
directly. The central question this paper deals with is: How does one train
deep neural nets whose final outputs are elements on a Riemannian manifold? To
answer this, we propose a general framework for manifold-aware training of deep
neural networks -- we utilize tangent spaces and exponential maps in order to
convert the proposed problem into a form that allows us to bring current
advances in deep learning to bear upon this problem. We describe two specific
applications to demonstrate this approach: prediction of probability
distributions for multi-class image classification, and prediction of
illumination-invariant subspaces from a single face-image via regression on the
Grassmannian. These applications show the generality of the proposed framework,
and result in improved performance over baselines that ignore the geometry of
the output space. In addition to solving this specific problem, we believe this
paper opens new lines of enquiry centered on the implications of Riemannian
geometry on deep architectures.
