  We address the problem of improving the performance and in particular the
sample complexity of deep neural networks by enforcing and guaranteeing
invariances to symmetry transformations rather than learning them from data.
Group-equivariant convolutions are a popular approach to obtain equivariant
representations. The desired corresponding invariance is then imposed using
pooling operations. For rotations, it has been shown that using invariant
integration instead of pooling further improves the sample complexity. In this
contribution, we first expand invariant integration beyond rotations to flips
and scale transformations. We then address the problem of incorporating
multiple desired invariances into a single network. For this purpose, we
propose a multi-stream architecture, where each stream is invariant to a
different transformation such that the network can simultaneously benefit from
multiple invariances. We demonstrate our approach with successful experiments
on Scaled-MNIST, SVHN, CIFAR-10 and STL-10.
