  In the field of numerical weather prediction (NWP), the probabilistic
distribution of the future state of the atmosphere is sampled with
Monte-Carlo-like simulations, called ensembles. These ensembles have
deficiencies (such as conditional biases) that can be corrected thanks to
statistical post-processing methods. Several ensembles exist and may be
corrected with different statistiscal methods. A further step is to combine
these raw or post-processed ensembles. The theory of prediction with expert
advice allows us to build combination algorithms with theoretical guarantees on
the forecast performance. This article adapts this theory to the case of
probabilistic forecasts issued as step-wise cumulative distribution functions
(CDF). The theory is applied to wind speed forecasting, by combining several
raw or post-processed ensembles, considered as CDFs. The second goal of this
study is to explore the use of two forecast performance criteria: the Continous
ranked probability score (CRPS) and the Jolliffe-Primo test. Comparing the
results obtained with both criteria leads to reconsidering the usual way to
build skillful probabilistic forecasts, based on the minimization of the CRPS.
Minimizing the CRPS does not necessarily produce reliable forecasts according
to the Jolliffe-Primo test. The Jolliffe-Primo test generally selects reliable
forecasts, but could lead to issuing suboptimal forecasts in terms of CRPS. It
is proposed to use both criterion to achieve reliable and skillful
probabilistic forecasts.
