  Recent studies have shown that massive quiescent galaxies at high redshift
are much more compact than present-day galaxies of the same mass. Here we
compare the radial stellar density profiles and the number density of a sample
of massive galaxies at z ~ 2.3 to nearby massive elliptical galaxies. We
confirm that the average stellar densities of the z ~ 2.3 galaxies within the
effective radius, rho(<r_e), are two orders of magnitude higher than those of
local elliptical galaxies of the same stellar mass. However, we also find that
the densities measured within a constant physical radius of 1 kpc, rho(<1 kpc),
are higher by a factor of 2-3 only. This suggests that inside-out growth
scenarios are plausible, in which the compact high redshift galaxies make up
the centers of normal nearby ellipticals. The compact galaxies are common at
high redshift, which enables us to further constrain their evolution by
requiring that the number density of their descendants does not exceed
constraints imposed by the z=0 galaxy mass function. We infer that size growth
must be efficient, with (r_{1+2}/r_1) ~ (M_{1+2}/M_1)^2. A simple model where
compact galaxies with masses ~ 10^11 Msun primarily grow through minor mergers
produces descendants with the approximate sizes, stellar densities, and number
density of elliptical galaxies with masses 2-3 x10^11 Msun in the local
Universe. We note that this model also predicts evolution in the M_BH - sigma
relation, such that the progenitors of elliptical galaxies have lower black
hole masses at fixed velocity dispersion. The main observational uncertainty is
the conversion from light to mass; measurements of kinematics are needed to
calibrate the masses and stellar densities of the high redshift galaxies.
