  The Hessian matrix has a number of important applications in a variety of
different fields, such as optimzation, image processing and statistics. In this
paper we focus on the practical aspects of efficiently computing Hessian
matrices in the context of deep learning using the Python scripting language
and the TensorFlow library. We define a general feed-forward neural network
model and show how to efficiently compute two quantities: the cost function's
exact Hessian matrix, and the cost function's approximate Hessian matrix, known
as the Outer Product of Gradients (OPG) matrix. Furthermore, as the number of
parameters (weights and biases) in deep learning usually is very large, we show
how to reduce the quadratic space complexity by an efficient implementation
based on approximate eigendecompositions.
