  Transformation groups, such as translations or rotations, effectively express
part of the variability observed in many recognition problems. The group
structure enables the construction of invariant signal representations with
appealing mathematical properties, where convolutions, together with pooling
operators, bring stability to additive and geometric perturbations of the
input. Whereas physical transformation groups are ubiquitous in image and audio
applications, they do not account for all the variability of complex signal
classes.
  We show that the invariance properties built by deep convolutional networks
can be cast as a form of stable group invariance. The network wiring
architecture determines the invariance group, while the trainable filter
coefficients characterize the group action. We give explanatory examples which
illustrate how the network architecture controls the resulting invariance
group. We also explore the principle by which additional convolutional layers
induce a group factorization enabling more abstract, powerful invariant
representations.
