  We study the impact of the nonlinear growth of the large-scale structure
(LSS) on the removal of the gravitational lensing effect (delensing) in cosmic
microwave background (CMB) $B$ modes. The importance of the nonlinear growth of
the LSS in the gravitational lensing analysis of CMB has been recently
recognized by several works, while its impact on delensing is not yet explored.
The delensing using mass-tracers such as galaxies and cosmic infrared
background (CIB) could be also affected by the nonlinear growth. We find that
the nonlinear growth of the LSS leads to $\sim 0.3\%$ corrections to $B$-mode
spectrum after delensing with a high-$z$ mass tracer ($z_m\sim 2$) at
$\ell=1000$-$2000$. The off-diagonal correlation coefficients of the lensing
$B$-mode template spectrum become significant for delensing with low-$z$
tracers ($z_m\lesssim 0.5$), but are negligible with high-$z$ tracers (such as
CIB). On the other hand, the power spectrum covariance of the delensed $B$ mode
is not significantly affected by the nonlinear growth of the LSS, and the
delensing efficiency is not significantly changed even if we use low-$z$
tracers. The CMB $B$-mode internal delensing is also not significantly affected
by the nonlinear growth.
