  Backpropagation is the workhorse of deep learning, however, several other
biologically-motivated learning rules have been introduced, such as random
feedback alignment and difference target propagation. None of these methods
have produced a competitive performance against backpropagation. In this paper,
we show that biologically-motivated learning rules with skip connections
between intermediate layers can perform as well as backpropagation on the MNIST
dataset and are robust to various sets of hyper-parameters.
