  Variational logistic regression is a popular method for approximate Bayesian
inference seeing wide-spread use in many areas of machine learning including:
Bayesian optimization, reinforcement learning and multi-instance learning to
name a few. However, due to the intractability of the Evidence Lower Bound,
authors have turned to the use of Monte Carlo, quadrature or bounds to perform
inference, methods which are costly or give poor approximations to the true
posterior.
  In this paper we introduce a new bound for the expectation of softplus
function and subsequently show how this can be applied to variational logistic
regression and Gaussian process classification. Unlike other bounds, our
proposal does not rely on extending the variational family, or introducing
additional parameters to ensure the bound is tight. In fact, we show that this
bound is tighter than the state-of-the-art, and that the resulting variational
posterior achieves state-of-the-art performance, whilst being significantly
faster to compute than Monte-Carlo methods.
