  Error backpropagation is an extremely effective algorithm for assigning
credit in artificial neural networks. However, weight updates under Backprop
depend on lengthy recursive computations and require separate output and error
messages -- features not shared by biological neurons, that are perhaps
unnecessary. In this paper, we revisit Backprop and the credit assignment
problem. We first decompose Backprop into a collection of interacting learning
algorithms; provide regret bounds on the performance of these sub-algorithms;
and factorize Backprop's error signals. Using these results, we derive a new
credit assignment algorithm for nonparametric regression, Kickback, that is
significantly simpler than Backprop. Finally, we provide a sufficient condition
for Kickback to follow error gradients, and show that Kickback matches
Backprop's performance on real-world regression benchmarks.
