  Uncertainty quantification is at the core of the reliability and robustness
of machine learning. In this paper, we provide a theoretical framework to
dissect the uncertainty, especially the \textit{epistemic} component, in deep
learning into \textit{procedural variability} (from the training procedure) and
\textit{data variability} (from the training data), which is the first such
attempt in the literature to our best knowledge. We then propose two approaches
to estimate these uncertainties, one based on influence function and one on
batching. We demonstrate how our approaches overcome the computational
difficulties in applying classical statistical methods. Experimental
evaluations on multiple problem settings corroborate our theory and illustrate
how our framework and estimation can provide direct guidance on modeling and
data collection efforts.
