  This Letter examines how the sizes, structures, and color gradients of
galaxies change along the quiescent sequence. Our sample consists of ~400
quiescent galaxies at $1.0\le z\le2.5$ and $10.1 \le \log{M_*/M_\odot}\le11.6$
in three CANDELS fields. We exploit deep multi-band HST imaging to derive
accurate mass profiles and color gradients, then use an empirical calibration
from rest-frame UVJ colors to estimate galaxy ages. We find that -- contrary to
previous results -- the youngest quiescent galaxies are not significantly
smaller than older quiescent galaxies at fixed stellar mass. These
`post-starburst' galaxies only appear smaller in half-light radii because they
have systematically flatter color gradients. The strength of color gradients in
quiescent galaxies is a clear function of age, with older galaxies exhibiting
stronger negative color gradients (i.e., redder centers). Furthermore, we find
that the central mass surface density $\Sigma_1$ is independent of age at fixed
stellar mass, and only weakly depends on redshift. This finding implies that
the central mass profiles of quiescent galaxies do not significantly change
with age; however, we find that older quiescent galaxies have additional mass
at large radii. Our results support the idea that building a massive core is a
necessary requirement for quenching beyond $z=1$, and indicate that
post-starburst galaxies are the result of a rapid quenching process that
requires structural change. Furthermore, our observed color gradient and mass
profile evolution supports a scenario where quiescent galaxies grow inside-out
via minor mergers.
