  We propose models to explain the hierarchies of the quark masses and mixing
by utilizing the $S_4^\prime$ modular flavor symmetry. The hierarchy is
realized by the modulus $\tau$ stabilized at $\mathrm{Im}\,\tau \gg 1$, where
the residual $Z_4^T$ symmetry is approximately unbroken and the
Froggatt-Nielsen mechanism works. It is found that the quark hierarchies are
realized only in a few cases of quark representations. We study two models with
assigning the modular weights, so that the observed quark hierarchies are
explained in the cases of both small and large ratios of the top to bottom
Yukawa couplings. We also argue that $\mathcal{O}({0.1})$ hierarchies of the
$\mathcal{O}({1})$ coefficients can be explained by imposing another $S_3$
modular symmetry.
