  We consider the problem of designing a sparse Gaussian process classifier
(SGPC) that generalizes well. Viewing SGPC design as constructing an additive
model like in boosting, we present an efficient and effective SGPC design
method to perform a stage-wise optimization of a predictive loss function. We
introduce new methods for two key components viz., site parameter estimation
and basis vector selection in any SGPC design. The proposed adaptive sampling
based basis vector selection method aids in achieving improved generalization
performance at a reduced computational cost. This method can also be used in
conjunction with any other site parameter estimation methods. It has similar
computational and storage complexities as the well-known information vector
machine and is suitable for large datasets. The hyperparameters can be
determined by optimizing a predictive loss function. The experimental results
show better generalization performance of the proposed basis vector selection
method on several benchmark datasets, particularly for relatively smaller basis
vector set sizes or on difficult datasets.
