  We study the structural evolution of massive galaxies by linking progenitors
and descendants at a constant cumulative number density of n_c=1.4x10^{-4}
Mpc^{-3} to z~3. Structural parameters were measured by fitting Sersic profiles
to high resolution CANDELS HST WFC3 J_{125} and H_{160} imaging in the
UKIDSS-UDS at 1<z<3 and ACS I_{814} imaging in COSMOS at 0.25<z<1. At a given
redshift, we selected the HST band that most closely samples a common
rest-frame wavelength so as to minimize systematics from color gradients in
galaxies. At fixed n_c, galaxies grow in stellar mass by a factor of ~3 from
z~3 to z~0. The size evolution is complex: galaxies appear roughly constant in
size from z~3 to z~2 and then grow rapidly to lower redshifts. The evolution in
the surface mass density profiles indicates that most of the mass at r<2 kpc
was in place by z~2, and that most of the new mass growth occurred at larger
radii. This inside-out mass growth is therefore responsible for the larger
sizes and higher Sersic indices of the descendants toward low redshift. At z<2,
the effective radius evolves with the stellar mass as r_e M^{2.0}, consistent
with scenarios that find dissipationless minor mergers to be a key driver of
size evolution. The progenitors at z~3 were likely star-forming disks with
r_e~2 kpc, based on their low Sersic index of n~1, low median axis ratio of
b/a~0.52, and typical location in the star-forming region of the U-V versus V-J
diagram. By z~1.5, many of these star-forming disks disappeared, giving rise to
compact quiescent galaxies. Toward lower redshifts, these galaxies continued to
assemble mass at larger radii and became the local ellipticals that dominate
the high mass end of the mass function at the present epoch.
