  Are multi-layer parameterized quantum circuits (MPQCs) more expressive than
classical neural networks (NNs)? How, why, and in what aspects? In this work,
we survey and develop intuitive insights into the expressive power of MPQCs in
relation to classical NNs. We organize available sources into a systematic
proof on why MPQCs are able to generate probability distributions that cannot
be efficiently simulated classically. We first show that instantaneous quantum
polynomial circuits (IQPCs), are unlikely to be simulated classically to within
a multiplicative error, and then show that MPQCs efficiently generalize IQPCs.
We support the surveyed claims with numerical simulations: with the MPQC as the
core architecture, we build different versions of quantum generative models to
learn a given multi-dimensional, multi-modal discrete data distribution, and
show their superior performances over a classical Generative Adversarial
Network (GAN) equipped with the Gumbel Softmax for generating discrete data. In
addition, we address practical issues such as how to efficiently train a
quantum circuit with only limited samples, how to efficiently calculate the
(quantum) gradient, and how to alleviate modal collapse. We propose and
experimentally verify an efficient training-and-fine-tuning scheme for lowering
the output noise and decreasing modal collapse. As an original contribution, we
develop a novel loss function (MCR loss) inspired by an information-theoretical
measure -- the coding rate reduction metric, which has a more expressive and
geometrically meaningful latent space representations -- beneficial for both
model selection and alleviating modal collapse. We derive the gradient of our
MCR loss with respect to the circuit parameters under two settings: with the
radial basis function (RBF) kernel and with a NN discriminator and conduct
experiments to showcase its effectiveness.
