  Building on recent work on capsule networks, we propose a new,
general-purpose form of "routing by agreement" that activates output capsules
in a layer as a function of their net benefit to use and net cost to ignore
input capsules from earlier layers. To illustrate the usefulness of our routing
algorithm, we present two capsule networks that apply it in different domains:
vision and language. The first network achieves new state-of-the-art accuracy
of 99.1% on the smallNORB visual recognition task with fewer parameters and an
order of magnitude less training than previous capsule models, and we find
evidence that it learns to perform a form of "reverse graphics." The second
network achieves new state-of-the-art accuracies on the root sentences of the
Stanford Sentiment Treebank: 58.5% on fine-grained and 95.6% on binary labels
with a single-task model that routes frozen embeddings from a pretrained
transformer as capsules. In both domains, we train with the same regime. Code
is available at https://github.com/glassroom/heinsen_routing along with
replication instructions.
