  We consider a model with three right-handed neutrinos in which Yukawa
coupling constants and Majorana masses are obtained by requiring the modular
$A_4$ symmetry. It has been shown that the model can explain mass hierarchies
and mixing patterns of charged leptons and neutrinos with the seesaw mechanism.
In this article we investigate the leptogenesis by decays of right-handed
neutrinos in this model. It is shown that masses of right-handed neutrinos are
about $10^{13}$ GeV in order to account for the observed baryon asymmetry of
the universe. Furthermore, the positive sign of the baryon asymmetry is
obtained only for the limited ranges of mixing angles and CP violation phases
of active neutrinos, which can be tested by future neutrino experiments.
