  We introduce a new Bayesian multi-class support vector machine by formulating
a pseudo-likelihood for a multi-class hinge loss in the form of a
location-scale mixture of Gaussians. We derive a variational-inference-based
training objective for gradient-based learning. Additionally, we employ an
inducing point approximation which scales inference to large data sets.
Furthermore, we develop hybrid Bayesian neural networks that combine standard
deep learning components with the proposed model to enable learning for
unstructured data. We provide empirical evidence that our model outperforms the
competitor methods with respect to both training time and accuracy in
classification experiments on 68 structured and two unstructured data sets.
Finally, we highlight the key capability of our model in yielding prediction
uncertainty for classification by demonstrating its effectiveness in the tasks
of large-scale active learning and detection of adversarial images.
