  Analog crossbar architectures for accelerating neural network training and
inference have made tremendous progress over the past several years. These
architectures are ideal for dense layers with fewer than roughly a thousand
neurons. However, for large sparse layers, crossbar architectures are highly
inefficient. A new hardware architecture, dubbed the MN3 (Memristive Nanowire
Neural Network), was recently described as an efficient architecture for
simulating very wide, sparse neural network layers, on the order of millions of
neurons per layer. The MN3 utilizes a high-density memristive nanowire mesh to
efficiently connect large numbers of silicon neurons with modifiable weights.
Here, in order to explore the MN3's ability to function as a deep neural
network, we describe one algorithm for training deep MN3 models and benchmark
simulations of the architecture on two deep learning tasks. We utilize a simple
piecewise linear memristor model, since we seek to demonstrate that training
is, in principle, possible for randomized nanowire architectures. In future
work, we intend on utilizing more realistic memristor models, and we will adapt
the presented algorithm appropriately. We show that the MN3 is capable of
performing composition, gradient propagation, and weight updates, which
together allow it to function as a deep neural network. We show that a
simulated multilayer perceptron (MLP), built from MN3 networks, can obtain a
1.61% error rate on the popular MNIST dataset, comparable to equivalently sized
software-based network. This work represents, to the authors' knowledge, the
first randomized nanowire architecture capable of reproducing the
backpropagation algorithm.
