  Ordinal regression is a classification task where classes have an order and
prediction error increases the further the predicted class is from the true
class. The standard approach for modeling ordinal data involves fitting
parallel separating hyperplanes that optimize a certain loss function. This
assumption offers sample efficient learning via inductive bias, but is often
too restrictive in real-world datasets where features may have varying effects
across different categories. Allowing class-specific hyperplane slopes creates
generalized logistic ordinal regression, increasing the flexibility of the
model at a cost to sample efficiency. We explore an extension of the
generalized model to the all-thresholds logistic loss and propose a
regularization approach that interpolates between these two extremes. Our
method, which we term continuously generalized ordinal logistic, significantly
outperforms the standard ordinal logistic model over a thorough set of ordinal
regression benchmark datasets. We further extend this method to deep learning
and show that it achieves competitive or lower prediction error compared to
previous models over a range of datasets and modalities. Furthermore, two
primary alternative models for deep learning ordinal regression are shown to be
special cases of our framework.
