  We constrain the evolution of the galaxy stellar mass function from 2 < z < 5
for galaxies with stellar masses as low as 10^5 Msun by combining star
formation histories of Milky Way satellite galaxies derived from deep Hubble
Space Telescope observations with merger trees from the ELVIS suite of N-body
simulations. This approach extends our understanding more than two orders of
magnitude lower in stellar mass than is currently possible by direct imaging.
We find the faint end slopes of the mass functions to be alpha=
-1.42(+0.07/-0.05) at z = 2 and alpha = -1.57^(+0.06/-0.06) at z = 5, and show
the slope only weakly evolves from z = 5 to z = 0. Our findings are in stark
contrast to a number of direct detection studies that suggest slopes as steep
as alpha = -1.9 at these epochs. Such a steep slope would result in an order of
magnitude too many luminous Milky Way satellites in a mass regime that is
observationally complete (Mstar > 2*10^5 Msun at z = 0). The most recent
studies from ZFOURGE and CANDELS also suggest flatter faint end slopes that are
consistent with our results, but with a lower degree of precision. This work
illustrates the strong connections between low and high-z observations when
viewed through the lens of LCDM numerical simulations.
