  We study quark and lepton mass matrices derived from magnetized
$T^2/\mathbb{Z}_2$ orbifold models. Quark and lepton masses have a large
hierarchy. In addition, mixing angles are large in the lepton sector, while
those are small in the quark sector. We find that this structure can be
realized in certain flavor models, which are identified by the zero points of
the zero-mode wave functions of fermions and Higgs modes. We classify such
realistic flavor models. Fixed points $\tau=i$, $e^{2\pi i/3}$ and $i\infty$ of
the modulus $\tau$ play a role in realizing a large mass hierarchy through our
scenario, where residual $S$, $ST$, and $T$ symmetries remain and the lightest
Higgs modes can correspond to eigenstates of residual symmetries at the leading
order. As a result, we find that there are 24 flavor models in total which can
be realistic in a vicinity of $S$-symmetric vacuum but no flavor models for
$ST$ and $T$-symmetric vacua.
