  We demonstrate a method to reduce number fluctuations in an ultracold atomic
sample using real-time feedback. By measuring the Faraday rotation of an
off-resonant probe laser beam with a pair of avalanche photodetectors in a
polarimetric setup we produce a proxy for the number of atoms in the sample. We
iteratively remove a fraction of the excess atoms from the sample to converge
on a target proxy value in a way that is insensitive to environmental
perturbations and robust to errors in light polarization. Using absorption
imaging for out-of-loop verification, we demonstrate a reduction in the number
fluctuations from $3\%$ to $0.45\%$ for samples at a temperature of 16.4 $\mu$K
over the time-scale of several hours which is limited by temperature
fluctuations, beam pointing noise, and photon shot noise.
