  We study the CP violation and the CP phase of quark mass matrices in modular
flavor symmetric models. The CP symmetry remains at $\tau = e^{2 \pi i/3}$ by a
combination of the $T$-symmetry of the modular symmetry. However, the
$T$-symmetry breaking may lead to the CP violation at the fixed point $\tau =
e^{2 \pi i/3}$. We study such a possibility in magnetized orbifold models as
examples of modular flavor symmetric models. These models, in general, have
more than one candidates for Higgs modes, while generic string
compactifications also lead to several Higgs modes. These Higgs modes have
different behaviors under the $T$-transformation. The light Higgs mode can be a
linear combination of those modes so as to lead to realistic quark mass
matrices. The CP phase of mass matrix does not appear in a certain case, which
is determined by the $T$-transformation behavior. Deviation from it is
important to realize the physical CP phase. We discuss an example leading to
non-vanishing CP phase at the fixed point $\tau = e^{2 \pi i/3}$.
