  Despite their remarkable performance, deep neural networks remain mostly
``black boxes'', suggesting inexplicability and hindering their wide
applications in fields requiring making rational decisions. Here we introduce
HOPE (High-order Polynomial Expansion), a method for expanding a network into a
high-order Taylor polynomial on a reference input. Specifically, we derive the
high-order derivative rule for composite functions and extend the rule to
neural networks to obtain their high-order derivatives quickly and accurately.
From these derivatives, we can then derive the Taylor polynomial of the neural
network, which provides an explicit expression of the network's local
interpretations. Numerical analysis confirms the high accuracy, low
computational complexity, and good convergence of the proposed method.
Moreover, we demonstrate HOPE's wide applications built on deep learning,
including function discovery, fast inference, and feature selection. The code
is available at https://github.com/HarryPotterXTX/HOPE.git.
