  Toward a deeper understanding on the inner work of deep neural networks, we
investigate CNN (convolutional neural network) using DCN (deconvolutional
network) and randomization technique, and gain new insights for the intrinsic
property of this network architecture. For the random representations of an
untrained CNN, we train the corresponding DCN to reconstruct the input images.
Compared with the image inversion on pre-trained CNN, our training converges
faster and the yielding network exhibits higher quality for image
reconstruction. It indicates there is rich information encoded in the random
features; the pre-trained CNN may discard information irrelevant for
classification and encode relevant features in a way favorable for
classification but harder for reconstruction. We further explore the property
of the overall random CNN-DCN architecture. Surprisingly, images can be
inverted with satisfactory quality. Extensive empirical evidence as well as
theoretical analysis are provided.
