  We propose MNPCA, a novel non-linear generalization of (2D)$^2${PCA}, a
classical linear method for the simultaneous dimension reduction of both rows
and columns of a set of matrix-valued data. MNPCA is based on optimizing over
separate non-linear mappings on the left and right singular spaces of the
observations, essentially amounting to the decoupling of the two sides of the
matrices. We develop a comprehensive theoretical framework for MNPCA by viewing
it as an eigenproblem in reproducing kernel Hilbert spaces. We study the
resulting estimators on both population and sample levels, deriving their
convergence rates and formulating a coordinate representation to allow the
method to be used in practice. Simulations and a real data example demonstrate
MNPCA's good performance over its competitors.
