  (Abridged) Statistical analysis of the unresolved light in the Hubble Deep
Field (HDF) strongly constrains possible sources of the optical Extragalactic
Background Light (EBL). To test for the statistical signature of previously
undetected sources, we estimate the auto, cross, and color correlations of the
``sky'' in the HDF that remains after masking objects brighter than I=30 mag.
This measurement yields the most stringent limits to date on small-scale
structure in the night sky; analysis of shallower imaging would be dominated by
galaxies now detected by the HDF. Unless there is a truly uniform optical
background, the mean EBL is likely to be within a small fraction of the surface
brightness from detected galaxies. No currently plausible sources of additional
EBL satisfy the constraints that they (1) would not have already been detected,
(2) contribute EBL comparable to that from detected galaxies, and (3) do not
produce EBL fluctuations in excess of the upper limits set by correlations in
the HDF.
