  Processing sets or other unordered, potentially variable-sized inputs in
neural networks is usually handled by aggregating a number of input tensors
into a single representation. While a number of aggregation methods already
exist from simple sum pooling to multi-head attention, they are limited in
their representational power both from theoretical and empirical perspectives.
On the search of a principally more powerful aggregation strategy, we propose
an optimization-based method called Equilibrium Aggregation. We show that many
existing aggregation methods can be recovered as special cases of Equilibrium
Aggregation and that it is provably more efficient in some important cases.
Equilibrium Aggregation can be used as a drop-in replacement in many existing
architectures and applications. We validate its efficiency on three different
tasks: median estimation, class counting, and molecular property prediction. In
all experiments, Equilibrium Aggregation achieves higher performance than the
other aggregation techniques we test.
