  The tunability of conductance states of various emerging non-volatile
memristive devices emulates the plasticity of biological synapses, making it
promising in the hardware realization of large-scale neuromorphic systems. The
inference of the neural network can be greatly accelerated by the vector-matrix
multiplication (VMM) performed within a crossbar array of memristive devices in
one step. Nevertheless, the implementation of the VMM needs complex peripheral
circuits and the complexity further increases since non-idealities of
memristive devices prevent precise conductance tuning (especially for the
online training) and largely degrade the performance of the deep neural
networks (DNNs). Here, we present an efficient online training method of the
memristive deep belief net (DBN). The proposed memristive DBN uses
stochastically binarized activations, reducing the complexity of peripheral
circuits, and uses the contrastive divergence (CD) based gradient descent
learning algorithm. The analog VMM and digital CD are performed separately in a
mixed-signal hardware arrangement, making the memristive DBN high immune to
non-idealities of synaptic devices. The number of write operations on
memristive devices is reduced by two orders of magnitude. The recognition
accuracy of 95%~97% can be achieved for the MNIST dataset using pulsed synaptic
behaviors of various memristive synaptic devices.
