  Mixtures of experts probabilistically divide the input space into regions,
where the assumptions of each expert, or conditional model, need only hold
locally. Combined with Gaussian process (GP) experts, this results in a
powerful and highly flexible model. We focus on alternative mixtures of GP
experts, which model the joint distribution of the inputs and targets
explicitly. We highlight issues of this approach in multi-dimensional input
spaces, namely, poor scalability and the need for an unnecessarily large number
of experts, degrading the predictive performance and increasing uncertainty. We
construct a novel model to address these issues through a nested partitioning
scheme that automatically infers the number of components at both levels.
Multiple response types are accommodated through a generalised GP framework,
while multiple input types are included through a factorised exponential family
structure. We show the effectiveness of our approach in estimating a
parsimonious probabilistic description of both synthetic data of increasing
dimension and an Alzheimer's challenge dataset.
