  We introduce a design strategy for neural network macro-architecture based on
self-similarity. Repeated application of a simple expansion rule generates deep
networks whose structural layouts are precisely truncated fractals. These
networks contain interacting subpaths of different lengths, but do not include
any pass-through or residual connections; every internal signal is transformed
by a filter and nonlinearity before being seen by subsequent layers. In
experiments, fractal networks match the excellent performance of standard
residual networks on both CIFAR and ImageNet classification tasks, thereby
demonstrating that residual representations may not be fundamental to the
success of extremely deep convolutional neural networks. Rather, the key may be
the ability to transition, during training, from effectively shallow to deep.
We note similarities with student-teacher behavior and develop drop-path, a
natural extension of dropout, to regularize co-adaptation of subpaths in
fractal architectures. Such regularization allows extraction of
high-performance fixed-depth subnetworks. Additionally, fractal networks
exhibit an anytime property: shallow subnetworks provide a quick answer, while
deeper subnetworks, with higher latency, provide a more accurate answer.
