  Intermediate-mass black holes (IMBHs: masses between $100 - 10^{6}
M_{\odot}$) historically comprise of an elusive population compared to
stellar-mass and supermassive BHs. Recently IMBHs have started to be observed
at the centers of low-mass galaxies. We perform cosmological hydrodynamical
simulations of $(2 h^{-1} ~ {\rm Mpc})^3$ comoving boxes and investigate the
growth and feedback of central IMBHs in dwarf galaxies (DGs). The earliest BHs
appear at $z \sim 18 - 25$, and grow thereafter by accreting gas and by merger
with other BHs. We find that, starting from $10^{2} M_{\odot}$, it is possible
to build up IMBHs of a few$\times 10^{5} - 10^{6} M_{\odot}$ by $z = 5$, when
the BHs are seeded in halos less massive than $4 \times 10^{7} M_{\odot}$. The
BH accretion rates increase with time, and reaches $\dot{M}_{\rm BH} = (0.2 -
0.8) \dot{M}_{\rm Edd}$ for the massive IMBHs by $z = 4$. The star formation
rate density (SFRD) evolution of the DGs (stellar mass $10^{5} - 10^{8}
M_{\odot}$) has a peak plateau between $z = 4 - 6$. Star formation is quenched
between $z = 9 - 4$. The SFRD is reduced by factors up to $3$, when the BHs
have grown to a few times $10^5 M_{\odot}$. Even in the presence of stronger
SN-driven mass ejection, the BHs continue to grow up to $z \sim 6$, sustained
by gas inflows driven by galaxy mergers and interactions in a cosmological
environment. Our conclusions, based on numerical simulation results, support
the scenario that early feedback from IMBHs in gas-rich DGs at $z = 5 - 8$ can
potentially solve several anomalies in the DG mass range within the concordance
$\Lambda$CDM cosmological scenario (Silk 2017). Our results suggest that IMBHs
at DG centers grow faster than their host galaxies in the early Universe, and
the resulting BH feedback turns the DGs and the BHs dormant.
