  We use the scattering network as a generic and fixed ini-tialization of the
first layers of a supervised hybrid deep network. We show that early layers do
not necessarily need to be learned, providing the best results to-date with
pre-defined representations while being competitive with Deep CNNs. Using a
shallow cascade of 1 x 1 convolutions, which encodes scattering coefficients
that correspond to spatial windows of very small sizes, permits to obtain
AlexNet accuracy on the imagenet ILSVRC2012. We demonstrate that this local
encoding explicitly learns invariance w.r.t. rotations. Combining scattering
networks with a modern ResNet, we achieve a single-crop top 5 error of 11.4% on
imagenet ILSVRC2012, comparable to the Resnet-18 architecture, while utilizing
only 10 layers. We also find that hybrid architectures can yield excellent
performance in the small sample regime, exceeding their end-to-end
counterparts, through their ability to incorporate geometrical priors. We
demonstrate this on subsets of the CIFAR-10 dataset and on the STL-10 dataset.
