  We discuss the quark mass matrices in the $A_4$ modular symmetry, where the
$A_4$ triplet of Higgs is introduced for each up-quark and down-quark sectors,
respectively. The model has six real parameters and two complex parameters in
addition to the modulus $\tau$. By inputting six quark masses and three CKM
mixing angles, we can predict the CP violation phase $\delta$ and the Jarlskog
invariant $J_{CP}$. The predicted ranges of $\delta$ and $J_{CP}$ are
consistent with the observed values. The absolute value of $V_{ub}$ is smaller
than $0.0043$, while $V_{cb}$ is lager than $0.0436$. In conclusion, our quark
mass matrices with the $A_4$ modular symmetry can reproduce the CKM mixing
matrix completely with observed quark masses.
