  This paper studies the prediction task of tensor-on-tensor regression in
which both covariates and responses are multi-dimensional arrays (a.k.a.,
tensors) across time with arbitrary tensor order and data dimension. Existing
methods either focused on linear models without accounting for possibly
nonlinear relationships between covariates and responses, or directly employed
black-box deep learning algorithms that failed to utilize the inherent tensor
structure. In this work, we propose a Factor Augmented Tensor-on-Tensor Neural
Network (FATTNN) that integrates tensor factor models into deep neural
networks. We begin with summarizing and extracting useful predictive
information (represented by the ``factor tensor'') from the complex structured
tensor covariates, and then proceed with the prediction task using the
estimated factor tensor as input of a temporal convolutional neural network.
The proposed methods effectively handle nonlinearity between complex data
structures, and improve over traditional statistical models and conventional
deep learning approaches in both prediction accuracy and computational cost. By
leveraging tensor factor models, our proposed methods exploit the underlying
latent factor structure to enhance the prediction, and in the meantime,
drastically reduce the data dimensionality that speeds up the computation. The
empirical performances of our proposed methods are demonstrated via simulation
studies and real-world applications to three public datasets. Numerical results
show that our proposed algorithms achieve substantial increases in prediction
accuracy and significant reductions in computational time compared to benchmark
methods.
