  We report measurements of isotope shifts for the five spinless Yb isotopes on
the $6s^2\,^1\textrm{S}_0 \rightarrow 5d6s\,^1\textrm{D}_2$ transition using
Doppler-free two-photon spectroscopy. We combine these data with existing
measurements on two transitions in Yb$^+$ [Phys. Rev. Lett. 125, 123002
(2020)], where deviation from King-plot linearity showed hints of a new bosonic
force carrier at the 3$\sigma$ level. The combined data strongly reduces the
significance of the new-physics signal. We show that the observed nonlinearity
in the joint Yb/Yb$^+$ King-plot analysis can be accounted for by the
deformation of the Yb nuclei.
