  We propose a new learning framework, signal propagation (sigprop), for
propagating a learning signal and updating neural network parameters via a
forward pass, as an alternative to backpropagation. In sigprop, there is only
the forward path for inference and learning. So, there are no structural or
computational constraints necessary for learning to take place, beyond the
inference model itself, such as feedback connectivity, weight transport, or a
backward pass, which exist under backpropagation based approaches. That is,
sigprop enables global supervised learning with only a forward path. This is
ideal for parallel training of layers or modules. In biology, this explains how
neurons without feedback connections can still receive a global learning
signal. In hardware, this provides an approach for global supervised learning
without backward connectivity. Sigprop by construction has compatibility with
models of learning in the brain and in hardware than backpropagation, including
alternative approaches relaxing learning constraints. We also demonstrate that
sigprop is more efficient in time and memory than they are. To further explain
the behavior of sigprop, we provide evidence that sigprop provides useful
learning signals in context to backpropagation. To further support relevance to
biological and hardware learning, we use sigprop to train continuous time
neural networks with Hebbian updates, and train spiking neural networks with
only the voltage or with biologically and hardware compatible surrogate
functions.
