  Convolutional neural networks (CNNs) achieve translational invariance by
using pooling operations. However, the operations do not preserve the spatial
relationships in the learned representations. Hence, CNNs cannot extrapolate to
various geometric transformations of inputs. Recently, Capsule Networks
(CapsNets) have been proposed to tackle this problem. In CapsNets, each entity
is represented by a vector and routed to high-level entity representations by a
dynamic routing algorithm. CapsNets have been shown to be more robust than CNNs
to affine transformations of inputs. However, there is still a huge gap between
their performance on transformed inputs compared to untransformed versions. In
this work, we first revisit the routing procedure by (un)rolling its forward
and backward passes. Our investigation reveals that the routing procedure
contributes neither to the generalization ability nor to the affine robustness
of the CapsNets. Furthermore, we explore the limitations of capsule
transformations and propose affine CapsNets (Aff-CapsNets), which are more
robust to affine transformations. On our benchmark task, where models are
trained on the MNIST dataset and tested on the AffNIST dataset, our
Aff-CapsNets improve the benchmark performance by a large margin (from 79% to
93.21%), without using any routing mechanism.
