  VHE (Very High Energy, E>100 GeV) radiation emitted at cosmological distances
will pair produce on low-energy diffuse extragalactic background radiation
before ever reaching us. This prevents us from directly seeing most of the VHE
emission in the Universe. However, a VHE gamma-ray that pair produces initiates
an electromagnetic pair cascade. At low energies, this secondary cascade
radiation has a spectrum insensitive to the spectrum of the primary
gamma-radiation and, unlike the original VHE radiation, IS observable.
Motivated by new measurements of the extragalactic MeV-GeV diffuse gamma-ray
background, we discuss the constraints placed on cosmological VHE source
populations by requiring that the cascade background they produce not exceed
the observed levels. We use a new, accurate cascading code and pay particular
attention to the dependence of the constraints on the diffuse cosmic background
at infrared/optical wavelengths. Despite considerable uncertainty in this
background, we find that robust constraints may still be placed on the
integrated emissivity of potential VHE sources in the Universe. The limits are
tighter than those obtained by considering cascading on the microwave
background alone and restrict significantly, for example, the parameter space
available for the exotic particle physics scenarios recently proposed to
explain the highest energy cosmic ray events. If direct emission from blazar
AGN in fact accounts for most of the observed GeV background, these limits
strengthen and rule out AGN emission scenarios which produce significant power
above 300 GeV.
