  Backpropagation has rapidly become the workhorse credit assignment algorithm
for modern deep learning methods. Recently, modified forms of predictive coding
(PC), an algorithm with origins in computational neuroscience, have been shown
to result in approximately or exactly equal parameter updates to those under
backpropagation. Due to this connection, it has been suggested that PC can act
as an alternative to backpropagation with desirable properties that may
facilitate implementation in neuromorphic systems. Here, we explore these
claims using the different contemporary PC variants proposed in the literature.
We obtain time complexity bounds for these PC variants which we show are
lower-bounded by backpropagation. We also present key properties of these
variants that have implications for neurobiological plausibility and their
interpretations, particularly from the perspective of standard PC as a
variational Bayes algorithm for latent probabilistic models. Our findings shed
new light on the connection between the two learning frameworks and suggest
that, in its current forms, PC may have more limited potential as a direct
replacement of backpropagation than previously envisioned.
