  Automatic differentiation (AD) is a technique for computing the derivative of
a function represented by a program. This technique is considered as the
de-facto standard for computing the differentiation in many machine learning
and optimisation software tools. Despite the practicality of this technique,
the performance of the differentiated programs, especially for functional
languages and in the presence of vectors, is suboptimal. We present an AD
system for a higher-order functional array-processing language. The core
functional language underlying this system simultaneously supports both
source-to-source forward-mode AD and global optimisations such as loop
transformations. In combination, gradient computation with forward-mode AD can
be as efficient as reverse mode, and the Jacobian matrices required for
numerical algorithms such as Gauss-Newton and Levenberg-Marquardt can be
efficiently computed.
