  Scattering networks are a class of designed Convolutional Neural Networks
(CNNs) with fixed weights. We argue they can serve as generic representations
for modelling images. In particular, by working in scattering space, we achieve
competitive results both for supervised and unsupervised learning tasks, while
making progress towards constructing more interpretable CNNs. For supervised
learning, we demonstrate that the early layers of CNNs do not necessarily need
to be learned, and can be replaced with a scattering network instead. Indeed,
using hybrid architectures, we achieve the best results with predefined
representations to-date, while being competitive with end-to-end learned CNNs.
Specifically, even applying a shallow cascade of small-windowed scattering
coefficients followed by 1$\times$1-convolutions results in AlexNet accuracy on
the ILSVRC2012 classification task. Moreover, by combining scattering networks
with deep residual networks, we achieve a single-crop top-5 error of 11.4% on
ILSVRC2012. Also, we show they can yield excellent performance in the small
sample regime on CIFAR-10 and STL-10 datasets, exceeding their end-to-end
counterparts, through their ability to incorporate geometrical priors. For
unsupervised learning, scattering coefficients can be a competitive
representation that permits image recovery. We use this fact to train hybrid
GANs to generate images. Finally, we empirically analyze several properties
related to stability and reconstruction of images from scattering coefficients.
