  Since radio sources with Ultra Steep Spectra (USS; alpha <~ -1.30; S ~
nu^alpha) are efficient tracers of high redshift radio galaxies (HzRGs), we
have defined three samples of such USS sources using the recently completed
WENSS, TEXAS, MRC, NVSS and PMN radio-surveys. Our combined sample contains 669
sources with S_1400 > 10 mJy and covers virtually the entire sky outside the
Galactic plane (|b|>15). For our 2 largest samples, covering declination > -35,
we selected only sources with angular sizes Theta < 1'. For 410 sources, we
present radio-maps with 0.3" to ~5" resolution from VLA and ATCA observations
or from the FIRST survey, which allows the optical identification of these
radio sources. We find that the spectral index distribution of 143,000 sources
from the WENSS and NVSS consists of a steep spectrum galaxy and a flat spectrum
quasar population, with the relative contribution of flat spectrum sources
doubling from S_1400 >0.1 Jy to S_1400 >2.5 Jy. The identification fraction of
our USS sources on the POSS (R <~ 20) is as low as 15%, independent of spectral
index alpha < -1.30. We further show that 85% of the USS sources that can be
identified with an X-ray source are probably contained in galaxy clusters, and
that alpha < -1.6 sources are excellent Galactic pulsar candidates, because the
percentage of these sources is four times higher in the Galactic plane. Our
sample has been constructed to start an intensive campaign to obtain a large
sample of high redshift objects (z>3) that is selected in a way that does not
suffer from dust extinction or any other optical bias [abridged].
