  Data-driven control in unknown environments requires a clear understanding of
the involved uncertainties for ensuring safety and efficient exploration. While
aleatoric uncertainty that arises from measurement noise can often be
explicitly modeled given a parametric description, it can be harder to model
epistemic uncertainty, which describes the presence or absence of training
data. The latter can be particularly useful for implementing exploratory
control strategies when system dynamics are unknown. We propose a novel method
for detecting the absence of training data using deep learning, which gives a
continuous valued scalar output between $0$ (indicating low uncertainty) and
$1$ (indicating high uncertainty). We utilize this detector as a proxy for
epistemic uncertainty and show its advantages over existing approaches on
synthetic and real-world datasets. Our approach can be directly combined with
aleatoric uncertainty estimates and allows for uncertainty estimation in
real-time as the inference is sample-free unlike existing approaches for
uncertainty modeling. We further demonstrate the practicality of this
uncertainty estimate in deploying online data-efficient control on a simulated
quadcopter acted upon by an unknown disturbance model.
