  Neural networks work remarkably well in practice and theoretically they can
be universal approximators. However, they still make mistakes and a specific
type of them called adversarial errors seem inexcusable to humans. In this
work, we analyze both test errors and adversarial errors on a well controlled
but highly non-linear visual classification problem. We find that, when
approximating training on infinite data, test errors tend to be close to the
ground truth decision boundary. Qualitatively speaking these are also more
difficult for a human. By contrast, adversarial examples can be found almost
everywhere and are often obvious mistakes. However, when we constrain
adversarial examples to the manifold, we observe a 90\% reduction in
adversarial errors. If we inflate the manifold by training with Gaussian noise
we observe a similar effect. In both cases, the remaining adversarial errors
tend to be close to the ground truth decision boundary. Qualitatively, the
remaining adversarial errors are similar to test errors on difficult examples.
They do not have the customary quality of being inexcusable mistakes.
