  In order to scale standard Gaussian process (GP) regression to large-scale
datasets, aggregation models employ factorized training process and then
combine predictions from distributed experts. The state-of-the-art aggregation
models, however, either provide inconsistent predictions or require
time-consuming aggregation process. We first prove the inconsistency of typical
aggregations using disjoint or random data partition, and then present a
consistent yet efficient aggregation model for large-scale GP. The proposed
model inherits the advantages of aggregations, e.g., closed-form inference and
aggregation, parallelization and distributed computing. Furthermore,
theoretical and empirical analyses reveal that the new aggregation model
performs better due to the consistent predictions that converge to the true
underlying function when the training size approaches infinity.
