  Experiments aiming to directly detect dark matter through particle recoils
can achieve energy thresholds of $\mathcal{O}(1\,\mathrm{eV})$. In this regime,
ionization signals from small-angle Compton scatters of environmental
$\gamma$-rays constitute a significant background. Monte Carlo simulations used
to build background models have not been experimentally validated at these low
energies. We report a precision measurement of Compton scattering on silicon
atomic shell electrons down to 23$\,$eV. A skipper charge-coupled device (CCD)
with single-electron resolution, developed for the DAMIC-M experiment, was
exposed to a $^{241}$Am $\gamma$-ray source over several months. Features
associated with the silicon K, L$_{1}$, and L$_{2,3}$-shells are clearly
identified, and scattering on valence electrons is detected for the first time
below 100$\,$eV. We find that the relativistic impulse approximation for
Compton scattering, which is implemented in Monte Carlo simulations commonly
used by direct detection experiments, does not reproduce the measured spectrum
below 0.5$\,$keV. The data are in better agreement with $ab$ $initio$
calculations originally developed for X-ray absorption spectroscopy.
