  Random forest is a popular prediction approach for handling high dimensional
covariates. However, it often becomes infeasible to interpret the obtained high
dimensional and non-parametric model. Aiming for obtaining an interpretable
predictive model, we develop a forward variable selection method using the
continuous ranked probability score (CRPS) as the loss function. Our stepwise
procedure leads to a smallest set of variables that optimizes the CRPS risk by
performing at each step a hypothesis test on a significant decrease in CRPS
risk. We provide mathematical motivation for our method by proving that in
population sense the method attains the optimal set. Additionally, we show that
the test is consistent provided that the random forest estimator of a quantile
function is consistent.
  In a simulation study, we compare the performance of our method with an
existing variable selection method, for different sample sizes and different
correlation strength of covariates. Our method is observed to have a much lower
false positive rate. We also demonstrate an application of our method to
statistical post-processing of daily maximum temperature forecasts in the
Netherlands. Our method selects about 10% covariates while retaining the same
predictive power.
