  We propose a novel distributional regression model for a multivariate
response vector based on a copula process over the covariate space. It uses the
implicit copula of a Gaussian multivariate regression, which we call a
``regression copula''. To allow for large covariate vectors their coefficients
are regularized using a novel multivariate extension of the horseshoe prior.
Bayesian inference and distributional predictions are evaluated using efficient
variational inference methods, allowing application to large datasets. An
advantage of the approach is that the marginal distributions of the response
vector can be estimated separately and accurately, resulting in predictive
distributions that are marginally-calibrated. Two substantive applications of
the methodology highlight its efficacy in multivariate modeling. The first is
the econometric modeling and prediction of half-hourly regional Australian
electricity prices. Here, our approach produces more accurate distributional
forecasts than leading benchmark methods. The second is the evaluation of
multivariate posteriors in likelihood-free inference (LFI) of a model for tree
species abundance data, extending a previous univariate regression copula LFI
method. In both applications, we demonstrate that our new approach exhibits a
desirable marginal calibration property.
