  Overparametrization is one of the most surprising and notorious phenomena in
machine learning. Recently, there have been several efforts to study if, and
how, Quantum Neural Networks (QNNs) acting in the absence of hardware noise can
be overparametrized. In particular, it has been proposed that a QNN can be
defined as overparametrized if it has enough parameters to explore all
available directions in state space. That is, if the rank of the Quantum Fisher
Information Matrix (QFIM) for the QNN's output state is saturated. Here, we
explore how the presence of noise affects the overparametrization phenomenon.
Our results show that noise can "turn on" previously-zero eigenvalues of the
QFIM. This enables the parametrized state to explore directions that were
otherwise inaccessible, thus potentially turning an overparametrized QNN into
an underparametrized one. For small noise levels, the QNN is
quasi-overparametrized, as large eigenvalues coexists with small ones. Then, we
prove that as the magnitude of noise increases all the eigenvalues of the QFIM
become exponentially suppressed, indicating that the state becomes insensitive
to any change in the parameters. As such, there is a pull-and-tug effect where
noise can enable new directions, but also suppress the sensitivity to parameter
updates. Finally, our results imply that current QNN capacity measures are
ill-defined when hardware noise is present.
