  The prevalence of neural networks in society is expanding at an increasing
rate. It is becoming clear that providing robust guarantees on systems that use
neural networks is very important, especially in safety-critical applications.
A trained neural network's sensitivity to adversarial attacks is one of its
greatest shortcomings. To provide robust guarantees, one popular method that
has seen success is to bound the activation functions using equality and
inequality constraints. However, there are numerous ways to form these bounds,
providing a trade-off between conservativeness and complexity. Depending on the
complexity of these bounds, the computational time of the optimisation problem
varies, with longer solve times often leading to tighter bounds. We approach
the problem from a different perspective, using sparse polynomial optimisation
theory and the Positivstellensatz, which derives from the field of real
algebraic geometry. The former exploits the natural cascading structure of the
neural network using ideas from chordal sparsity while the later asserts the
emptiness of a semi-algebraic set with a nested family of tests of
non-decreasing accuracy to provide tight bounds. We show that bounds can be
tightened significantly, whilst the computational time remains reasonable. We
compare the solve times of different solvers and show how the accuracy can be
improved at the expense of increased computation time. We show that by using
this sparse polynomial framework the solve time and accuracy can be improved
over other methods for neural network verification with ReLU, sigmoid and tanh
activation functions.
