  Accurate disturbance estimation is essential for safe robot operations. The
recently proposed neural moving horizon estimation (NeuroMHE), which uses a
portable neural network to model the MHE's weightings, has shown promise in
further pushing the accuracy and efficiency boundary. Currently, NeuroMHE is
trained through gradient descent, with its gradient computed recursively using
a Kalman filter. This paper proposes a trust-region policy optimization method
for training NeuroMHE. We achieve this by providing the second-order
derivatives of MHE, referred to as the MHE Hessian. Remarkably, we show that
much of computation already used to obtain the gradient, especially the Kalman
filter, can be efficiently reused to compute the MHE Hessian. This offers
linear computational complexity relative to the MHE horizon. As a case study,
we evaluate the proposed trust region NeuroMHE on real quadrotor flight data
for disturbance estimation. Our approach demonstrates highly efficient training
in under 5 min using only 100 data points. It outperforms a state-of-the-art
neural estimator by up to 68.1% in force estimation accuracy, utilizing only
1.4% of its network parameters. Furthermore, our method showcases enhanced
robustness to network initialization compared to the gradient descent
counterpart.
