  In this work we study the properties of deep neural networks (DNN) with
random weights. We formally prove that these networks perform a
distance-preserving embedding of the data. Based on this we then draw
conclusions on the size of the training data and the networks' structure. A
longer version of this paper with more results and details can be found in
(Giryes et al., 2015). In particular, we formally prove in the longer version
that DNN with random Gaussian weights perform a distance-preserving embedding
of the data, with a special treatment for in-class and out-of-class data.
