  We introduce a simple permutation equivariant layer for deep learning with
set structure.This type of layer, obtained by parameter-sharing, has a simple
implementation and linear-time complexity in the size of each set. We use deep
permutation-invariant networks to perform point-could classification and
MNIST-digit summation, where in both cases the output is invariant to
permutations of the input. In a semi-supervised setting, where the goal is make
predictions for each instance within a set, we demonstrate the usefulness of
this type of layer in set-outlier detection as well as semi-supervised learning
with clustering side-information.
