  Deep neural networks are in the limelight of machine learning with their
excellent performance in many data-driven applications. However, they can lead
to inaccurate predictions when queried in out-of-distribution data points,
which can have detrimental effects especially in sensitive domains, such as
healthcare and transportation, where erroneous predictions can be very costly
and/or dangerous. Subsequently, quantifying the uncertainty of the output of a
neural network is often leveraged to evaluate the confidence of its
predictions, and ensemble models have proved to be effective in measuring the
uncertainty by utilizing the variance of predictions over a pool of models. In
this paper, we propose a novel approach for uncertainty quantification via
ensembles, called Random Activation Functions (RAFs) Ensemble, that aims at
improving the ensemble diversity toward a more robust estimation, by
accommodating each neural network with a different (random) activation
function. Extensive empirical study demonstrates that RAFs Ensemble outperforms
state-of-the-art ensemble uncertainty quantification methods on both synthetic
and real-world datasets in a series of regression tasks.
