  A restricted Boltzmann machine (RBM) learns a probability distribution over
its input samples and has numerous uses like dimensionality reduction,
classification and generative modeling. Conventional RBMs accept vectorized
data that dismisses potentially important structural information in the
original tensor (multi-way) input. Matrix-variate and tensor-variate RBMs,
named MvRBM and TvRBM, have been proposed but are all restrictive by model
construction, which leads to a weak model expression power. This work presents
the matrix product operator RBM (MPORBM) that utilizes a tensor network
generalization of Mv/TvRBM, preserves input formats in both the visible and
hidden layers, and results in higher expressive power. A novel training
algorithm integrating contrastive divergence and an alternating optimization
procedure is also developed. Numerical experiments compare the MPORBM with the
traditional RBM and MvRBM for data classification and image completion and
denoising tasks. The expressive power of the MPORBM as a function of the
MPO-rank is also investigated.
