  Gradient-based first-order adaptive optimization methods such as the Adam
optimizer are prevalent in training artificial networks, achieving the
state-of-the-art results. This work attempts to answer the question whether it
is viable for biological neural systems to adopt such optimization methods. To
this end, we demonstrate a realization of the Adam optimizer using
biologically-plausible mechanisms in synapses. The proposed learning rule has
clear biological correspondence, runs continuously in time, and achieves
performance to comparable Adam's. In addition, we present a new approach,
inspired by the predisposition property of synapses observed in neuroscience,
to circumvent the biological implausibility of the weight transport problem in
backpropagation (BP). With only local information and no separate training
phases, this method establishes and maintains weight symmetry in the forward
and backward signaling paths, and is applicable to the proposed biologically
plausible Adam learning rule. These mechanisms may shed light on the way in
which biological synaptic dynamics facilitate learning.
