  The recent advances in machine learning in various fields of applications can
be largely attributed to the rise of deep learning (DL) methods and
architectures. Despite being a key technology behind autonomous cars, image
processing, speech recognition, etc., a notorious problem remains the lack of
theoretical understanding of DL and related interpretability and (adversarial)
robustness issues. Understanding the specifics of DL, as compared to, say,
other forms of nonlinear regression methods or statistical learning, is
interesting from a mathematical perspective, but at the same time it is of
crucial importance in practice: treating neural networks as mere black boxes
might be sufficient in certain cases, but many applications require waterproof
performance guarantees and a deeper understanding of what could go wrong and
why it could go wrong. It is probably fair to say that, despite being
mathematically well founded as a method to approximate complicated functions,
DL is mostly still more like modern alchemy that is firmly in the hands of
engineers and computer scientists. Nevertheless, it is evident that certain
specifics of DL that could explain its success in applications demands
systematic mathematical approaches. In this work, we review robustness issues
of DL and particularly bridge concerns and attempts from approximation theory
to statistical learning theory. Further, we review Bayesian Deep Learning as a
means for uncertainty quantification and rigorous explainability.
