  In this paper, we propose an analytical method for performing tractable
approximate Gaussian inference (TAGI) in Bayesian neural networks. The method
enables the analytical Gaussian inference of the posterior mean vector and
diagonal covariance matrix for weights and biases. The method proposed has a
computational complexity of $\mathcal{O}(n)$ with respect to the number of
parameters $n$, and the tests performed on regression and classification
benchmarks confirm that, for a same network architecture, it matches the
performance of existing methods relying on gradient backpropagation.
