  Convolutional and Recurrent, deep neural networks have been successful in
machine learning systems for computer vision, reinforcement learning, and other
allied fields. However, the robustness of such neural networks is seldom
apprised, especially after high classification accuracy has been attained. In
this paper, we evaluate the robustness of three recurrent neural networks to
tiny perturbations, on three widely used datasets, to argue that high accuracy
does not always mean a stable and a robust (to bounded perturbations,
adversarial attacks, etc.) system. Especially, normalizing the spectrum of the
discrete recurrent network to bound the spectrum (using power method, Rayleigh
quotient, etc.) on a unit disk produces stable, albeit highly non-robust neural
networks. Furthermore, using the $\epsilon$-pseudo-spectrum, we show that
training of recurrent networks, say using gradient-based methods, often result
in non-normal matrices that may or may not be diagonalizable. Therefore, the
open problem lies in constructing methods that optimize not only for accuracy
but also for the stability and the robustness of the underlying neural network,
a criterion that is distinct from the other.
