  It is common practice to use Laplace approximations to compute marginal
likelihoods in Bayesian versions of generalised linear models (GLM). Marginal
likelihoods combined with model priors are then used in different search
algorithms to compute the posterior marginal probabilities of models and
individual covariates. This allows performing Bayesian model selection and
model averaging. For large sample sizes, even the Laplace approximation becomes
computationally challenging because the optimisation routine involved needs to
evaluate the likelihood on the full set of data in multiple iterations. As a
consequence, the algorithm is not scalable for large datasets. To address this
problem, we suggest using a version of a popular batch stochastic gradient
descent (BSGD) algorithm for estimating the marginal likelihood of a GLM by
subsampling from the data. We further combine the algorithm with Markov chain
Monte Carlo (MCMC) based methods for Bayesian model selection and provide some
theoretical results on the convergence of the estimates. Finally, we report
results from experiments illustrating the performance of the proposed
algorithm.
