  Many high-dimensional and large-volume data sets of practical relevance have
hierarchical structures induced by trees, graphs or time series. Such data sets
are hard to process in Euclidean spaces and one often seeks low-dimensional
embeddings in other space forms to perform required learning tasks. For
hierarchical data, the space of choice is a hyperbolic space since it
guarantees low-distortion embeddings for tree-like structures. Unfortunately,
the geometry of hyperbolic spaces has properties not encountered in Euclidean
spaces that pose challenges when trying to rigorously analyze algorithmic
solutions. Here, for the first time, we establish a unified framework for
learning scalable and simple hyperbolic linear classifiers with provable
performance guarantees. The gist of our approach is to focus on Poincar\'e ball
models and formulate the classification problems using tangent space
formalisms. Our results include a new hyperbolic and second-order perceptron
algorithm as well as an efficient and highly accurate convex optimization setup
for hyperbolic support vector machine classifiers. All algorithms provably
converge and are highly scalable as they have complexities comparable to those
of their Euclidean counterparts. Their performance accuracies on synthetic data
sets comprising millions of points, as well as on complex real-world data sets
such as single-cell RNA-seq expression measurements, CIFAR10, Fashion-MNIST and
mini-ImageNet.
