  The back-propagation (BP) algorithm has been considered the de-facto method
for training deep neural networks. It back-propagates errors from the output
layer to the hidden layers in an exact manner using the transpose of the
feedforward weights. However, it has been argued that this is not biologically
plausible because back-propagating error signals with the exact incoming
weights are not considered possible in biological neural systems. In this work,
we propose a biologically plausible paradigm of neural architecture based on
related literature in neuroscience and asymmetric BP-like methods.
Specifically, we propose two bidirectional learning algorithms with trainable
feedforward and feedback weights. The feedforward weights are used to relay
activations from the inputs to target outputs. The feedback weights pass the
error signals from the output layer to the hidden layers. Different from other
asymmetric BP-like methods, the feedback weights are also plastic in our
framework and are trained to approximate the forward activations. Preliminary
results show that our models outperform other asymmetric BP-like methods on the
MNIST and the CIFAR-10 datasets.
