  In this paper, we propose a dimensionality reduction method applied to
tensor-structured data as a hidden layer (we call it TensorProjection Layer) in
a convolutional neural network. Our proposed method transforms input tensors
into ones with a smaller dimension by projection. The directions of projection
are viewed as training parameters associated with our proposed layer and
trained via a supervised learning criterion such as minimization of the
cross-entropy loss function. We discuss the gradients of the loss function with
respect to the parameters associated with our proposed layer. We also implement
simple numerical experiments to evaluate the performance of the
TensorProjection Layer.
