  Many applications require robustness, or ideally invariance, of neural
networks to certain transformations of input data. Most commonly, this
requirement is addressed by training data augmentation, using adversarial
training, or defining network architectures that include the desired invariance
by design. In this work, we propose a method to make network architectures
provably invariant with respect to group actions by choosing one element from a
(possibly continuous) orbit based on a fixed criterion. In a nutshell, we
intend to 'undo' any possible transformation before feeding the data into the
actual network. Further, we empirically analyze the properties of different
approaches which incorporate invariance via training or architecture, and
demonstrate the advantages of our method in terms of robustness and
computational efficiency. In particular, we investigate the robustness with
respect to rotations of images (which can hold up to discretization artifacts)
as well as the provable orientation and scaling invariance of 3D point cloud
classification.
