  We present HOIMotion - a novel approach for human motion forecasting during
human-object interactions that integrates information about past body poses and
egocentric 3D object bounding boxes. Human motion forecasting is important in
many augmented reality applications but most existing methods have only used
past body poses to predict future motion. HOIMotion first uses an
encoder-residual graph convolutional network (GCN) and multi-layer perceptrons
to extract features from body poses and egocentric 3D object bounding boxes,
respectively. Our method then fuses pose and object features into a novel
pose-object graph and uses a residual-decoder GCN to forecast future body
motion. We extensively evaluate our method on the Aria digital twin (ADT) and
MoGaze datasets and show that HOIMotion consistently outperforms
state-of-the-art methods by a large margin of up to 8.7% on ADT and 7.2% on
MoGaze in terms of mean per joint position error. Complementing these
evaluations, we report a human study (N=20) that shows that the improvements
achieved by our method result in forecasted poses being perceived as both more
precise and more realistic than those of existing methods. Taken together,
these results reveal the significant information content available in
egocentric 3D object bounding boxes for human motion forecasting and the
effectiveness of our method in exploiting this information.
