  Deep neural networks have long training and processing times. Early exits
added to neural networks allow the network to make early predictions using
intermediate activations in the network in time-sensitive applications.
However, early exits increase the training time of the neural networks. We
introduce QuickNets: a novel cascaded training algorithm for faster training of
neural networks. QuickNets are trained in a layer-wise manner such that each
successive layer is only trained on samples that could not be correctly
classified by the previous layers. We demonstrate that QuickNets can
dynamically distribute learning and have a reduced training cost and inference
cost compared to standard Backpropagation. Additionally, we introduce
commitment layers that significantly improve the early exits by identifying for
over-confident predictions and demonstrate its success.
