  We analyze how passive galaxies at z $\sim$ 1.5 populate the mass-size plane
as a function of their stellar age, to understand if the observed size growth
with time can be explained with the appearance of larger quenched galaxies at
lower redshift. We use a sample of 32 passive galaxies extracted from the Wide
Field Camera 3 Infrared Spectroscopic Parallel (WISP) survey with spectroscopic
redshift 1.3 $\lesssim$ z $\lesssim$ 2.05, specific star-formation rates lower
than 0.01 Gyr$^{-1}$, and stellar masses above 4.5 $\times$ 10$^{10}$
M$_\odot$. All galaxies have spectrally determined stellar ages from fitting of
their rest-frame optical spectra and photometry with stellar population models.
When dividing our sample into young (age $\leq$ 2.1 Gyr) and old (age $>$ 2.1
Gyr) galaxies we do not find a significant trend in the distributions of the
difference between the observed radius and the one predicted by the mass-size
relation. This result indicates that the relation between the galaxy age and
its distance from the mass-size relation, if it exists, is rather shallow, with
a slope alpha $\gtrsim$ -0.6. At face value, this finding suggests that
multiple dry and/or wet minor mergers, rather than the appearance of newly
quenched galaxies, are mainly responsible for the observed time evolution of
the mass-size relation in passive galaxies.
