  Several recent simulations of galaxy formation predict two main phases of
supermassive black hole (BH) accretion: an early, highly intermittent phase
(during which BHs are under-massive relative to local scaling relations),
followed by a phase of accelerated growth. We investigate physical factors that
drive the transition in BH accretion in cosmological zoom-in simulations from
the FIRE project, ranging from dwarf galaxies to galaxies sufficiently massive
to host luminous quasars. The simulations model multi-channel stellar feedback,
but neglect AGN feedback. We show that multiple physical properties, including
halo mass, galaxy stellar mass, and depth of the central gravitational
potential correlate with accelerated BH fueling: constant thresholds in these
properties are typically crossed within ~0.1 Hubble time of accelerated BH
fueling. Black hole masses increase sharply when the stellar surface density in
the inner 1 kpc crosses a threshold Sigma1 ~ 10^9.5 Msun/kpc^2, a
characteristic value above which gravity prevents stellar feedback from
ejecting gas, and similar to the value above which galaxies are observed to
quench. We further show that accelerated BH growth correlates with the
emergence of long-lived, thin gas disks, as well as with virialization of the
inner circumgalactic medium. The halo mass Mh ~ 10^12 Msun and stellar mass
Mstar ~ 10^10.5 Msun at which BH growth accelerates correspond to ~L* galaxies.
The fact that stellar feedback becomes inefficient at ejecting gas from the
nucleus above this mass scale may play an important role in explaining why AGN
feedback appears to be most important in galaxies above ~L*.
