  Achieving robustness against adversarial input perturbation is an important
and intriguing problem in machine learning. In the area of semantic image
segmentation, a number of adversarial training approaches have been proposed as
a defense against adversarial perturbation, but the methodology of evaluating
the robustness of the models is still lacking, compared to image
classification. Here, we demonstrate that, just like in image classification,
it is important to evaluate the models over several different and hard attacks.
We propose a set of gradient based iterative attacks and show that it is
essential to perform a large number of iterations. We include attacks against
the internal representations of the models as well. We apply two types of
attacks: maximizing the error with a bounded perturbation, and minimizing the
perturbation for a given level of error. Using this set of attacks, we show for
the first time that a number of models in previous work that are claimed to be
robust are in fact not robust at all. We then evaluate simple adversarial
training algorithms that produce reasonably robust models even under our set of
strong attacks. Our results indicate that a key design decision to achieve any
robustness is to use only adversarial examples during training. However, this
introduces a trade-off between robustness and accuracy.
