  Segmentation models have been found to be vulnerable to targeted and
non-targeted adversarial attacks. However, the resulting segmentation outputs
are often so damaged that it is easy to spot an attack. In this paper, we
propose semantically stealthy adversarial attacks which can manipulate targeted
labels while preserving non-targeted labels at the same time. One challenge is
making semantically meaningful manipulations across datasets and models.
Another challenge is avoiding damaging non-targeted labels. To solve these
challenges, we consider each input image as prior knowledge to generate
perturbations. We also design a special regularizer to help extract features.
To evaluate our model's performance, we design three basic attack types, namely
`vanishing into the context,' `embedding fake labels,' and `displacing target
objects.' Our experiments show that our stealthy adversarial model can attack
segmentation models with a relatively high success rate on Cityscapes,
Mapillary, and BDD100K. Our framework shows good empirical generalization
across datasets and models.
