  Resistive random access memories (RRAM) are novel nonvolatile memory
technologies, which can be embedded at the core of CMOS, and which could be
ideal for the in-memory implementation of deep neural networks. A particularly
exciting vision is using them for implementing Binarized Neural Networks
(BNNs), a class of deep neural networks with a highly reduced memory footprint.
The challenge of resistive memory, however, is that they are prone to device
variation, which can lead to bit errors. In this work we show that BNNs can
tolerate these bit errors to an outstanding level, through simulations of
networks on the MNIST and CIFAR10 tasks. If a standard BNN is used, up to 10^-4
bit error rate can be tolerated with little impact on recognition performance
on both MNIST and CIFAR10. We then show that by adapting the training procedure
to the fact that the BNN will be operated on error-prone hardware, this
tolerance can be extended to a bit error rate of 4x10^-2. The requirements for
RRAM are therefore a lot less stringent for BNNs than more traditional
applications. We show, based on experimental measurements on a RRAM HfO2
technology, that this result can allow reduce RRAM programming energy by a
factor 30.
