  [Abridged] We present optical and NIR observations of 19 short GRB host
galaxies, aimed at measuring their stellar masses and population ages. The
goals of this study are to evaluate whether short GRBs track the stellar mass
distribution of galaxies, to investigate the progenitor delay time
distribution, and to explore any connection between long and short GRB
progenitors. Using single stellar population models we infer masses of
log(M/M_sun)=8.8-11.6 and population ages of tau=0.03-4.4 Gyr. We further infer
maximal masses of log(M/M_sun)=9.7-11.9 by assuming stellar population ages
equal to the age of the universe at each host's redshift. Comparing the
distribution of stellar masses to the general galaxy mass function we find that
short GRBs track the cosmic stellar mass distribution only if the late-type
hosts generally have maximal masses. However, there is an apparent dearth of
early-type hosts compared to the equal contribution of early- and late-type
galaxies to the cosmic stellar mass budget. These results suggest that stellar
mass may not be the sole parameter controlling the short GRB rate, and raise
the possibility of a two-component model with both mass and star formation
playing a role. If short GRBs in late-type galaxies indeed track the star
formation activity, the resulting typical delay time is ~0.2 Gyr, while those
in early-type hosts have a typical delay of ~3 Gyr. Using the same stellar
population models we fit the data for 22 long GRB hosts and find that they have
lower masses and younger population ages, with <log(M/M_sun)>=9.1 and
<tau>=0.06 Gyr, respectively; their maximal masses are similarly lower,
<log(M/M_sun)>=9.6. Most importantly, the two host populations remain distinct
even if we consider only the star-forming hosts of short GRBs, supporting our
previous findings that the progenitors of long GRBs and short GRBs in late-type
galaxies are distinct.
