  We quantify a limitation in the usual accounting of the finite-size effects,
where the leading $[(Z\alpha)^4]$ and subleading $[(Z\alpha)^5]$ contributions
to the Lamb shift are given by the mean-square radius and the third Zemach
moment of the charge distribution. In the presence of any non-smooth behaviour
of the nuclear form factor at scales comparable to the inverse Bohr radius, the
expansion of the Lamb shift in the moments breaks down. This is relevant for
some of the explanations of the "proton size puzzle". We find, for instance,
that the de R\'ujula toy model of the proton form factor does not resolve the
puzzle as claimed, despite the large value of the third Zemach moment. Without
relying on the radii expansion, we show how tiny, milli-percent (pcm) changes
in the proton electric form factor at a MeV scale would be able to explain the
puzzle. It shows that one needs to know all the soft contributions to proton
electric form factor to pcm accuracy for a precision extraction of the proton
charge radius from atomic Lamb shifts.
