  We propose a novel, succinct, and effective approach for distribution
prediction to quantify uncertainty in machine learning. It incorporates
adaptively flexible distribution prediction of
$\mathbb{P}(\mathbf{y}|\mathbf{X}=x)$ in regression tasks. This conditional
distribution's quantiles of probability levels spreading the interval $(0,1)$
are boosted by additive models which are designed by us with intuitions and
interpretability. We seek an adaptive balance between the structural integrity
and the flexibility for $\mathbb{P}(\mathbf{y}|\mathbf{X}=x)$, while Gaussian
assumption results in a lack of flexibility for real data and highly flexible
approaches (e.g., estimating the quantiles separately without a distribution
structure) inevitably have drawbacks and may not lead to good generalization.
This ensemble multi-quantiles approach called EMQ proposed by us is totally
data-driven, and can gradually depart from Gaussian and discover the optimal
conditional distribution in the boosting. On extensive regression tasks from
UCI datasets, we show that EMQ achieves state-of-the-art performance comparing
to many recent uncertainty quantification methods. Visualization results
further illustrate the necessity and the merits of such an ensemble model.
