  We combine the recent determination of the evolution of the cosmic density of
molecular gas (H_2) using deep, volumetric surveys, with previous estimates of
the cosmic density of stellar mass, star formation rate and atomic gas (HI), to
constrain the evolution of baryons associated with galaxies averaged over
cosmic time and space. The cosmic HI and H_2 densities are roughly equal at
z~1.5. The H_2 density then decreases by a factor 6^{+3}_{-2} to today's value,
whereas the HI density stays approximately constant. The stellar mass density
is increasing continuously with time and surpasses that of the total gas
density (HI and H_2) at redshift z~1.5. The growth in stellar mass cannot be
accounted for by the decrease in cosmic H_2 density, necessitating significant
accretion of additional gas onto galaxies. With the new H_2 constraints, we
postulate and put observational constraints on a two step gas accretion
process: (i) a net infall of ionized gas from the intergalactic/circumgalactic
medium to refuel the extended HI reservoirs, and (ii) a net inflow of HI and
subsequent conversion to H_2 in the galaxy centers. Both the infall and inflow
rate densities have decreased by almost an order of magnitude since z~2.
Assuming that the current trends continue, the cosmic molecular gas density
will further decrease by about a factor of two over the next 5 Gyr, the stellar
mass will increase by approximately 10%, and cosmic star formation activity
will decline steadily toward zero, as the gas infall and accretion shut down.
