  Predictive models for binary data are fundamental in various fields, and the
growing complexity of modern applications has motivated several flexible
specifications for modeling the relationship between the observed predictors
and the binary responses. A widely-implemented solution is to express the
probability parameter via a probit mapping of a Gaussian process indexed by
predictors. However, unlike for continuous settings, there is a lack of
closed-form results for predictive distributions in binary models with Gaussian
process priors. Markov chain Monte Carlo methods and approximation strategies
provide common solutions to this problem, but state-of-the-art algorithms are
either computationally intractable or inaccurate in moderate-to-high
dimensions. In this article, we aim to cover this gap by deriving closed-form
expressions for the predictive probabilities in probit Gaussian processes that
rely either on cumulative distribution functions of multivariate Gaussians or
on functionals of multivariate truncated normals. To evaluate these quantities
we develop novel scalable solutions based on tile-low-rank Monte Carlo methods
for computing multivariate Gaussian probabilities, and on mean-field
variational approximations of multivariate truncated normals. Closed-form
expressions for the marginal likelihood and for the posterior distribution of
the Gaussian process are also discussed. As shown in simulated and real-world
empirical studies, the proposed methods scale to dimensions where
state-of-the-art solutions are impractical.
