  This article provides an overview of some of the mathematical principles of
Automatic Differentiation (AD). In particular, we summarise different
descriptions of the Forward Mode of AD, like the matrix-vector product based
approach, the idea of lifting functions to the algebra of dual numbers, the
method of Taylor series expansion on dual numbers and the application of the
push-forward operator, and explain why they all reduce to the same actual chain
of computations. We further give a short mathematical description of some
methods of higher-order Forward AD and, at the end of this paper, briefly
describe the Reverse Mode of Automatic Differentiation.
