  Quantum neural networks (QNNs) offer a powerful paradigm for programming
near-term quantum computers and have the potential to speedup applications
ranging from data science to chemistry to materials science. However, a
possible obstacle to realizing that speedup is the Barren Plateau (BP)
phenomenon, whereby the gradient vanishes exponentially in the system size $n$
for certain QNN architectures. The question of whether high-order derivative
information such as the Hessian could help escape a BP was recently posed in
the literature. Here we show that the elements of the Hessian are exponentially
suppressed in a BP, so estimating the Hessian in this situation would require a
precision that scales exponentially with $n$. Hence, Hessian-based approaches
do not circumvent the exponential scaling associated with BPs. We also show the
exponential suppression of higher order derivatives. Hence, BPs will impact
optimization strategies that go beyond (first-order) gradient descent. In
deriving our results, we prove novel, general formulas that can be used to
analytically evaluate any high-order partial derivative on quantum hardware.
These formulas will likely have independent interest and use for training
quantum neural networks (outside of the context of BPs).
