  We show that unsupervised training of latent capsule layers using only the
reconstruction loss, without masking to select the correct output class, causes
a loss of equivariances and other desirable capsule qualities. This implies
that supervised capsules networks can't be very deep. Unsupervised sparsening
of latent capsule layer activity both restores these qualities and appears to
generalize better than supervised masking, while potentially enabling deeper
capsules networks. We train a sparse, unsupervised capsules network of similar
geometry to Sabour et al (2017) on MNIST, and then test classification accuracy
on affNIST using an SVM layer. Accuracy is improved from benchmark 79% to 90%.
