  Forecasting the full distribution of the number of earthquakes is revealed to
be inherently superior to forecasting their mean. Forecasting the full
distribution of earthquake numbers is also shown to yield robust projections in
the presence of "surprise" large earthquakes, which in the past have strongly
deteriorated the scores of existing models. We show this with
pseudo-prospective experiments on synthetic as well as real data from the
Advanced National Seismic System (ANSS) database for California, with
earthquakes with magnitude larger than 2.95 that occurred between the period
1971-2016. Our results call in question the testing methodology of the
Collaboratory for the study of earthquake predictability (CSEP), which amounts
to assuming a Poisson distribution of earthquake numbers, which is known to be
a poor representation of the heavy-tailed distribution of earthquake numbers.
Using a spatially varying ETAS model, we demonstrate a remarkable stability of
the forecasting performance, when using the full distribution of earthquake
numbers for the forecasts, even in the presence of large earthquakes such as Mw
7.1 Hector Mine, Mw 7.2 El Mayor-Cucapah, Mw 6.6 Sam Simeon earthquakes, or in
the presence of intense swarm activity in Northwest Nevada in 2014. While our
results have been derived for ETAS type models, we propose that all earthquake
forecasting models of any type should embrace the full distribution of
earthquake numbers, such that their true forecasting potential is revealed.
