  We trained three Binarized Convolutional Neural Network architectures
(LeNet-4, Network-In-Network, AlexNet) on a variety of datasets (MNIST,
CIFAR-10, CIFAR-100, extended SVHN, ImageNet) using error-prone activations and
tested them without errors to study the resilience of the training process.
With the exception of the AlexNet when trained on the ImageNet dataset, we
found that Bit Error Rates of a few percent during training do not degrade the
test accuracy. Furthermore, by training the AlexNet on progressively smaller
subsets of ImageNet classes, we observed increasing tolerance to activation
errors. The ability to operate with high BERs is critical for reducing power
consumption in existing hardware and for facilitating emerging memory
technologies. We discuss how operating at moderate BER can enable
Magnetoresistive RAM with higher endurance, speed and density.
