  The Collaboratory for the Study of Earthquake Predictability (CSEP) aims to
prospectively test time-dependent earthquake probability forecasts on their
consistency with observations. To compete, time-dependent seismicity models are
calibrated on earthquake catalog data. But catalogs contain much observational
uncertainty. We study the impact of magnitude uncertainties on rate estimates
in clustering models, on their forecasts and on their evaluation by CSEP's
consistency tests. First, we quantify magnitude uncertainties. We find that
magnitude uncertainty is more heavy-tailed than a Gaussian, such as a
double-sided exponential distribution, with scale parameter nu_c=0.1 - 0.3.
Second, we study the impact of such noise on the forecasts of a simple
clustering model which captures the main ingredients of popular short term
models. We prove that the deviations of noisy forecasts from an exact forecast
are power law distributed in the tail with exponent alpha=1/(a*nu_c), where a
is the exponent of the productivity law of aftershocks. We further prove that
the typical scale of the fluctuations remains sensitively dependent on the
specific catalog. Third, we study how noisy forecasts are evaluated in CSEP
consistency tests. Noisy forecasts are rejected more frequently than expected
for a given confidence limit. The Poisson assumption of the consistency tests
is inadequate for short-term forecast evaluations. To capture the
idiosyncrasies of each model together with any propagating uncertainties, the
forecasts need to specify the entire likelihood distribution of seismic rates.
