  The proposed article aims at offering a comprehensive tutorial for the
computational aspects of structured matrix and tensor factorization. Unlike
existing tutorials that mainly focus on {\it algorithmic procedures} for a
small set of problems, e.g., nonnegativity or sparsity-constrained
factorization, we take a {\it top-down} approach: we start with general
optimization theory (e.g., inexact and accelerated block coordinate descent,
stochastic optimization, and Gauss-Newton methods) that covers a wide range of
factorization problems with diverse constraints and regularization terms of
engineering interest. Then, we go `under the hood' to showcase specific
algorithm design under these introduced principles. We pay a particular
attention to recent algorithmic developments in structured tensor and matrix
factorization (e.g., random sketching and adaptive step size based stochastic
optimization and structure-exploiting second-order algorithms), which are the
state of the art---yet much less touched upon in the literature compared to
{\it block coordinate descent} (BCD)-based methods. We expect that the article
to have an educational values in the field of structured factorization and hope
to stimulate more research in this important and exciting direction.
