  The introduction of convolutional layers greatly advanced the performance of
neural networks on image tasks due to innately capturing a way of encoding and
learning translation-invariant operations, matching one of the underlying
symmetries of the image domain. In comparison, there are a number of problems
in which there are a number of different inputs which are all 'of the same
type' --- multiple particles, multiple agents, multiple stock prices, etc. The
corresponding symmetry to this is permutation symmetry, in that the algorithm
should not depend on the specific ordering of the input data. We discuss a
permutation-invariant neural network layer in analogy to convolutional layers,
and show the ability of this architecture to learn to predict the motion of a
variable number of interacting hard discs in 2D. In the same way that
convolutional layers can generalize to different image sizes, the permutation
layer we describe generalizes to different numbers of objects.
