  We use deep Chandra X-ray imaging to measure the distribution of specific
black hole accretion rates ($L_X$ relative to the stellar mass of the galaxy)
and thus trace AGN activity within star-forming and quiescent galaxies, as a
function of stellar mass (from $10^{8.5}-10^{11.5} M_\odot$) and redshift (to
$z \sim 4$). We adopt near-infrared selected samples of galaxies from the
CANDELS and UltraVISTA surveys, extract X-ray data for every galaxy, and use a
flexible Bayesian method to combine these data and to measure the probability
distribution function of specific black hole accretion rates,
$\lambda_{sBHAR}$. We identify a broad distribution of $\lambda_{sBHAR}$ in
both star-forming and quiescent galaxies---likely reflecting the stochastic
nature of AGN fuelling---with a roughly power-law shape that rises toward lower
$\lambda_{sBHAR}$, a steep cutoff at $\lambda_{sBHAR} \gtrsim 0.1-1$ (in
Eddington equivalent units), and a turnover or flattening at $\lambda_{sBHAR}
\lesssim 10^{-3}-10^{-2}$. We find that the probability of a star-forming
galaxy hosting a moderate $\lambda_{sBHAR}$ AGN depends on stellar mass and
evolves with redshift, shifting toward higher $\lambda_{sBHAR}$ at higher
redshifts. This evolution is truncated at a point corresponding to the
Eddington limit, indicating black holes may self-regulate their growth at high
redshifts when copious gas is available. The probability of a quiescent galaxy
hosting an AGN is generally lower than that of a star-forming galaxy, shows
signs of suppression at the highest stellar masses, and evolves strongly with
redshift. The AGN duty cycle in high-redshift ($z\gtrsim2$) quiescent galaxies
thus reaches $\sim$20 per cent, comparable to the duty cycle in star-forming
galaxies of equivalent stellar mass and redshift.
