  Theoretical studies have proven that the Hilbert space has remarkable
performance in many fields of applications. Frames in tensor product of Hilbert
spaces were introduced to generalize the inner product to high-order tensors.
However, these techniques require tensor decomposition which could lead to the
loss of information and it is a NP-hard problem to determine the rank of
tensors. Here, we present a new framework, namely matrix Hilbert space to
perform a matrix inner product space when data observations are represented as
matrices. We preserve the structure of initial data and multi-way correlation
among them is captured in the process. In addition, we extend the reproducing
kernel Hilbert space (RKHS) to reproducing kernel matrix Hilbert space (RKMHS)
and propose an equivalent condition of the space uses of the certain kernel
function. A new family of kernels is introduced in our framework to apply the
classifier of Support Tensor Machine(STM) and comparative experiments are
performed on a number of real-world datasets to support our contributions.
