  Additive-interactive regression has recently been shown to offer attractive
minimax error rates over traditional nonparametric multivariate regression in a
wide variety of settings, including cases where the predictor count is much
larger than the sample size and many of the predictors have important effects
on the response, potentially through complex interactions. We present a
Bayesian implementation of additive-interactive regression using an additive
Gaussian process (AGP) prior and develop an efficient Markov chain sampler that
extends stochastic search variable selection in this setting. Careful prior and
hyper-parameter specification are developed in light of performance and
computational considerations, and key innovations address difficulties in
exploring a joint posterior distribution over multiple subsets of high
dimensional predictor inclusion vectors. The method offers state-of-the-art
support and interaction recovery while improving dramatically over competitors
in terms of prediction accuracy on a diverse set of simulated and real data.
Results from real data studies provide strong evidence that the
additive-interactive framework is an attractive modeling platform for
high-dimensional nonparametric regression.
