  We theoretically develop and experimentally demonstrate a holographic method
for imaging cold atoms at the diffraction and photon shot noise limits. Aided
by a double point source reference field, a simple iterative algorithm robustly
removes the twin image of an $^{87}$Rb cold atom sample during the image
reconstruction. Shot-noise limited phase shift and absorption images are
consistently retrieved at various probe detunings, and during the laser cooling
process. We consistently resolve less than 2 mrad phase shift ($0.4~\%$
attenuation) of the probe light, outperforming shot-noise limited
phase-contrast (absorption) imaging by a factor of 4 or more if the same camera
is used without pixel saturation. We discuss the possible extension of this
work for precise phase imaging of dense atomic gases, and for off-resonant
probing of multiple atoms in optical lattices.
