  We simulate propagation of cosmic ray nucleons above 10^{19} eV in scenarios
where both the source distribution and magnetic fields within about 50 Mpc from
us are obtained from an unconstrained large scale structure simulation. We find
that consistency of predicted sky distributions with current data above 4 x
10^{19} eV requires magnetic fields of ~0.1 microGauss in our immediate
environment, and a nearby source density of ~10^{-4}-10^{-3} Mpc^{-3}. Radio
galaxies could provide the required sources, but only if both high and
low-luminosity radio galaxies are very efficient cosmic ray accelerators.
Moreover, at ~10^{19} eV an additional isotropic flux component, presumably of
cosmological origin, should dominate over the local flux component by about a
factor three in order to explain the observed isotropy. This argues against the
scenario in which local astrophysical sources of cosmic rays above ~10^{19} eV
reside in strongly magnetized (B~0.1 microGauss) and structured intergalactic
medium. Finally we discuss how future large scale full-sky detectors such as
the Pierre Auger project will allow to put much more stringent constraints on
source and magnetic field distributions.
