  This paper presents a deep learning enhanced adaptive unscented Kalman filter
(UKF) for predicting human arm motion in the context of manufacturing. Unlike
previous network-based methods that solely rely on captured human motion data,
which is represented as bone vectors in this paper, we incorporate a human arm
dynamic model into the motion prediction algorithm and use the UKF to
iteratively forecast human arm motions. Specifically, a
Lagrangian-mechanics-based physical model is employed to correlate arm motions
with associated muscle forces. Then a Recurrent Neural Network (RNN) is
integrated into the framework to predict future muscle forces, which are
transferred back to future arm motions based on the dynamic model. Given the
absence of measurement data for future human motions that can be input into the
UKF to update the state, we integrate another RNN to directly predict human
future motions and treat the prediction as surrogate measurement data fed into
the UKF. A noteworthy aspect of this study involves the quantification of
uncertainties associated with both the data-driven and physical models in one
unified framework. These quantified uncertainties are used to dynamically adapt
the measurement and process noises of the UKF over time. This adaption, driven
by the uncertainties of the RNN models, addresses inaccuracies stemming from
the data-driven model and mitigates discrepancies between the assumed and true
physical models, ultimately enhancing the accuracy and robustness of our
predictions. Compared to the traditional RNN-based prediction, our method
demonstrates improved accuracy and robustness in extensive experimental
validations of various types of human motions.
