  We study the relationship between the morphology and star formation history
(SFH) of 361 quiescent galaxies (QGs) at redshift $\langle
z_{obs}\rangle\approx 2$, with stellar mass $\log M_*\ge10.3$, selected with
the UVJ technique. Taking advantage of panchromatic photometry covering the
rest-frame UV-to-NIR spectral range ($\approx40$ bands), we reconstruct the
non-parametric SFH of the galaxies with the fully Bayesian SED fitting code
Prospector. We find that the half-light radius $R_e$, observed at $z_{obs}$,
depends on the formation redshift of the galaxies, $z_{form}$, and that this
relationship depends on stellar mass. At $\log M_*<11$, the relationship is
consistent with $R_e\propto(1+z_{form})^{-1}$, in line with the expectation
that the galaxies' central density depends on the cosmic density at the time of
their formation, i.e. the "progenitor effect". At $\log M_*>11$, the
relationship between $R_e$ and $z_{form}$ flattens, suggesting that mergers
become increasingly important for the size growth of more massive galaxies
after they quenched. We also find that the relationship between $z_{form}$ and
galaxy compactness similarly depends on stellar mass. While no clear trend is
observed for QGs with $\log M_*>11$, lower-mass QGs that formed earlier, i.e.
with larger $z_{form}$, have larger central stellar mass surface densities,
both within the $R_e$ ($\Sigma_e$) and central 1 kpc ($\Sigma_{1kpc}$), and
also larger $M_{1kpc}/M_*$, the fractional mass within the central 1 kpc. These
trends between $z_{form}$ and compactness, however, essentially disappear, if
the progenitor effect is removed by normalizing the stellar density with the
cosmic density at $z_{form}$. Our findings highlight the importance of
reconstructing the SFH of galaxies before attempting to infer their intrinsic
structural evolution.
