  The two most important algorithms in artificial intelligence are
backpropagation and belief propagation. In spite of their importance, the
connection between them is poorly characterized. We show that when an input to
backpropagation is converted into an input to belief propagation so that
(loopy) belief propagation can be run on it, then the result of belief
propagation encodes the result of backpropagation; thus backpropagation is
recovered as a special case of belief propagation. In other words, we prove for
apparently the first time that belief propagation generalizes backpropagation.
Our analysis is a theoretical contribution, which we motivate with the
expectation that it might reconcile our understandings of each of these
algorithms, and serve as a guide to engineering researchers seeking to improve
the behavior of systems that use one or the other.
