  Recent advances in deep neural network demand more than millions of
parameters to handle and mandate the high-performance computing resources with
improved efficiency. The cross-bar array architecture has been considered as
one of the promising deep learning architectures that shows a significant
computing gain over the conventional processors. To investigate the feasibility
of the architecture, we examine non-idealities and their impact on the
performance. Specifically, we study the impact of failed cells due to the
initialization process of the resistive memory based cross-bar array. Unlike
the conventional memory array, individual memory elements cannot be rerouted
and, thus, may have a critical impact on model accuracy. We categorize the
possible failures and propose hardware implementation that minimizes
catastrophic failures. Such hardware optimization bounds the possible logical
value of the failed cells and gives us opportunities to compensate for the loss
of accuracy via off-line training. By introducing the random weight defects
during the training, we show that the model becomes more resilient on the
device initialization failures, therefore, less prone to degrade the inference
performance due to the failed devices. Our study sheds light on the hardware
and software co-optimization procedure to cope with potentially catastrophic
failures in the cross-bar array.
