  A large lepton number asymmetry of $\mathcal{O}(0.1-1)$ at present universe
might not only be allowed but also necessary for consistency among cosmological
data. We show that, if a sizeable lepton number asymmetry were produced before
the electroweak phase transition, the requirement for not producing too much
baryon number asymmetry through sphalerons processes, forces the high scale
lepton number asymmetry to be larger than about $30$. Therefore a mild entropy
release causing $\mathcal{O}(10-100)$ suppression of pre-existing particle
density should take place, when the background temperature of the universe is
around $T = \mathcal{O}(10^{-2} - 10^2) {\rm GeV}$ for a large but
experimentally consistent asymmetry to be present today. We also show that such
a mild entropy production can be obtained by the late-time decays of the
saxion, constraining the parameters of the Peccei-Quinn sector such as the mass
and the vacuum expectation value of the saxion field to be $m_\phi \gtrsim
\mathcal{O}(10) {\rm TeV}$ and $\phi_0 \gtrsim \mathcal{O}(10^{14}) {\rm GeV}$,
respectively.
