  We study the CP-violating phase of the quark sector in the $U(8)$ flavor
model on $T^2/Z_N \, (N=2,3,4,6)$ with non-vanishing magnetic fluxes, where
properties of possible origins of the CP violation are investigated minutely.
In this system, a non-vanishing value is mandatory in the real part of the
complex modulus parameter $\tau$ of the two-dimensional torus. On $T^2$ without
orbifolding, underlying discrete flavor symmetries severely restrict the form
of Yukawa couplings and it is very difficult to reproduce the observed pattern
in the quark sector including the CP-violating phase $\delta_{\rm CP}$. In
cases of multiple Higgs doublets emerging on $T^2/Z_2$, the mass matrices of
the zero-mode fermions can be written in the Gaussian textures by choosing
appropriate configurations of vacuum expectation values of the Higgs fields.
When such Gaussian textures of mass matrices are realized, we show that all of
the quark profiles, which are mass hierarchies among the quarks, quark mixing
angles, and $\delta_{\rm CP}$ can be simultaneously realized.
