  We investigate the origin of the evolution of the population-averaged central
stellar mass density ($\Sigma_1$) of quiescent galaxies (QGs) by probing the
relation between stellar age and $\Sigma_1$ at $z\sim0$. We use the Zurich
ENvironmental Study (ZENS), which is a survey of galaxy groups with a large
fraction of satellite galaxies. QGs shape a narrow locus in the
$\Sigma_1-M_{\star}$ plane, which we refer to as $\Sigma_1$ ridgeline. Colors
of ($B-I$) and ($I-J$) are used to divide QGs into three age categories: young
($<2~\mathrm{Gyr}$), intermediate ($2-4~\mathrm{Gyr}$), and old
($>4~\mathrm{Gyr}$). At fixed stellar mass, old QGs on the $\Sigma_1$ ridgeline
have higher $\Sigma_1$ than young QGs. This shows that galaxies landing on the
$\Sigma_1$ ridgeline at later epochs arrive with lower $\Sigma_1$, which drives
the zeropoint of the ridgeline down with time. We compare the present-day
zeropoint of the oldest population at $z=0$ with the zeropoint of the quiescent
population 4 Gyr back in time, at $z=0.37$. These zeropoints are identical,
showing that the intrinsic evolution of individual galaxies after they arrive
on the $\Sigma_1$ ridgeline must be negligible, or must evolve parallel to the
ridgeline during this interval. The observed evolution of the global zeropoint
of 0.07 dex over the last 4 Gyr is thus largely due to the continuous addition
of newly quenched galaxies with lower $\Sigma_1$ at later times ("progenitor
bias"). While these results refer to the satellite-rich ZENS sample as a whole,
our work suggests a similar age-$\Sigma_1$ trend for central galaxies.
