  Skew-Gaussian processes (SkewGPs) extend the multivariate Unified Skew-Normal
distributions over finite dimensional vectors to distribution over functions.
SkewGPs are more general and flexible than Gaussian processes, as SkewGPs may
also represent asymmetric distributions. In a recent contribution we showed
that SkewGP and probit likelihood are conjugate, which allows us to compute the
exact posterior for non-parametric binary classification and preference
learning. In this paper, we generalize previous results and we prove that
SkewGP is conjugate with both the normal and affine probit likelihood, and more
in general, with their product. This allows us to (i) handle classification,
preference, numeric and ordinal regression, and mixed problems in a unified
framework; (ii) derive closed-form expression for the corresponding posterior
distributions. We show empirically that the proposed framework based on SkewGP
provides better performance than Gaussian processes in active learning and
Bayesian (constrained) optimization. These two tasks are fundamental for design
of experiments and in Data Science.
