  We propose an $A_4$ modular invariant flavor model of leptons, in which both
CP and modular symmetries are broken spontaneously by the vacuum expectation
value of the modulus $\tau$. The value of the modulus $\tau$ is restricted by
the observed lepton mixing angles and lepton masses for the normal hierarchy of
neutrino masses at $3\sigma$ confidence level. The predictive Dirac CP phase
$\delta_{CP}$ is in the ranges $[0^\circ,50^\circ]$, $[170^\circ,175^\circ]$
and $[280^\circ,360^\circ]$ for ${\rm Re}\, [\tau]<0$, and
$[0^\circ,80^\circ]$, $[185^\circ,190^\circ]$ and $[310^\circ,360^\circ]$ for
${\rm Re}\,[\tau]>0$ at $3\sigma$ confidence level. The sum of three neutrino
masses is predicted in $[60, 84]$ meV, and the effective mass for the
$0\nu\beta\beta$ decay is in [0.003, 3] meV. On the other hand, there is no
allowed region of the modulus $\tau$ for the inverted hierarchy of neutrino
masses at $3\sigma$ confidence level. The modulus $\tau$ links the Dirac CP
phase to the cosmological baryon asymmetry (BAU) via the leptogenesis. Due to
the strong wash-out effect, the predictive baryon asymmetry $Y_B$ can be at
most the same order of the observed value. Then, the lightest right-handed
neutrino mass is restricted in the range of $M_1 =[1.5,\,6.5] \times 10^{13}$
GeV. We find the correlation between the predictive $Y_B$ and the Dirac CP
phase $\delta_{CP}$. Only two predictive $\delta_{CP}$ ranges,
$[5^\circ,40^\circ]$ (${\rm Re}\,[\tau]>0$) and $[320^\circ,355^\circ]$ (${\rm
Re}\,[\tau]<0$) are consistent with the BAU.
