  To perform adversarial attacks in the physical world, many studies have
proposed adversarial camouflage, a method to hide a target object by applying
camouflage patterns on 3D object surfaces. For obtaining optimal physical
adversarial camouflage, previous studies have utilized the so-called neural
renderer, as it supports differentiability. However, existing neural renderers
cannot fully represent various real-world transformations due to a lack of
control of scene parameters compared to the legacy photo-realistic renderers.
In this paper, we propose the Differentiable Transformation Attack (DTA), a
framework for generating a robust physical adversarial pattern on a target
object to camouflage it against object detection models with a wide range of
transformations. It utilizes our novel Differentiable Transformation Network
(DTN), which learns the expected transformation of a rendered object when the
texture is changed while preserving the original properties of the target
object. Using our attack framework, an adversary can gain both the advantages
of the legacy photo-realistic renderers including various physical-world
transformations and the benefit of white-box access by offering
differentiability. Our experiments show that our camouflaged 3D vehicles can
successfully evade state-of-the-art object detection models in the
photo-realistic environment (i.e., CARLA on Unreal Engine). Furthermore, our
demonstration on a scaled Tesla Model 3 proves the applicability and
transferability of our method to the real world.
