  In recent times, neural networks have become a powerful tool for the analysis
of complex and abstract data models. However, their introduction intrinsically
increases our uncertainty about which features of the analysis are
model-related and which are due to the neural network. This means that
predictions by neural networks have biases which cannot be trivially
distinguished from being due to the true nature of the creation and observation
of data or not. In order to attempt to address such issues we discuss Bayesian
neural networks: neural networks where the uncertainty due to the network can
be characterised. In particular, we present the Bayesian statistical framework
which allows us to categorise uncertainty in terms of the ingrained randomness
of observing certain data and the uncertainty from our lack of knowledge about
how data can be created and observed. In presenting such techniques we show how
errors in prediction by neural networks can be obtained in principle, and
provide the two favoured methods for characterising these errors. We will also
describe how both of these methods have substantial pitfalls when put into
practice, highlighting the need for other statistical techniques to truly be
able to do inference when using neural networks.
