  We study models of lepton masses and mixing based on broken modular
invariance. We consider invariance under the finite modular group $\Gamma_4
\simeq S_4$ and focus on the minimal scenario where the expectation value of
the modulus is the only source of symmetry breaking, such that no flavons need
to be introduced. After constructing a basis for the lowest weight modular
forms, we build two minimal models, one of which successfully accommodates
charged lepton masses and neutrino oscillation data, while predicting the
values of the Dirac and Majorana CPV phases.
