  Non-negative matrix factorization with the generalized Kullback-Leibler
divergence (NMF) and latent Dirichlet allocation (LDA) are two popular
approaches for dimensionality reduction of non-negative data. Here, we show
that NMF with $\ell_1$ normalization constraints on the columns of both
matrices of the decomposition and a Dirichlet prior on the columns of one
matrix is equivalent to LDA. To show this, we demonstrate that explicitly
accounting for the scaling ambiguity of NMF by adding $\ell_1$ normalization
constraints to the optimization problem allows a joint update of both matrices
in the widely used multiplicative updates (MU) algorithm. When both of the
matrices are normalized, the joint MU algorithm leads to probabilistic latent
semantic analysis (PLSA), which is LDA without a Dirichlet prior. Our approach
of deriving joint updates for NMF also reveals that a Lasso penalty on one
matrix together with an $\ell_1$ normalization constraint on the other matrix
is insufficient to induce any sparsity.
