  Extrasolar planets with sizes between that of the Earth and Neptune ($R_{\rm
p}=1{-}4~{\rm R}_\oplus$) have a bimodal radius distribution. This 'planet
radius valley' separates compact, rocky super-Earths ($R_{\rm p}=1.0{-}1.8~{\rm
R}_\oplus$) from larger sub-Neptunes ($R_{\rm p}=1.8{-}3.5~{\rm R}_\oplus$)
hosting a gaseous hydrogen-helium envelope around their rocky core. Various
hypotheses for this radius valley have been put forward, which all rely on
physics internal to the planetary system: photoevaporation by the host star,
long-term mass loss driven by the cooling planetary core, or the transition
between two fundamentally different planet formation modes as gas is lost from
the protoplanetary disc. Here we report the discovery that the planet radius
distribution exhibits a strong dependence on ambient stellar clustering,
characterised by measuring the position-velocity phase space density with
\textit{Gaia}. When dividing the planet sample into 'field' and 'overdensity'
sub-samples, we find that planetary systems in the field exhibit a
statistically significant ($p=5.5\times10^{-3}$) dearth of planets below the
radius valley compared to systems in phase space overdensities. This implies
that the large-scale stellar environment of a planetary system is a key factor
setting the planet radius distribution. We discuss how models for the radius
valley might be revised following our findings and conclude that a multi-scale,
multi-physics scenario is needed, connecting planet formation and evolution,
star and stellar cluster formation, and galaxy evolution.
