  Top quarks, produced in large numbers at the Large Hadron Collider, have a
complex detector signature and require special reconstruction techniques. The
most common decay mode, the "all-jet" channel, results in a 6-jet final state
which is particularly difficult to reconstruct in $pp$ collisions due to the
large number of permutations possible. We present a novel approach to this
class of problem, based on neural networks using a generalized attention
mechanism, that we call Symmetry Preserving Attention Networks (SPA-Net). We
train one such network to identify the decay products of each top quark
unambiguously and without combinatorial explosion as an example of the power of
this technique.This approach significantly outperforms existing
state-of-the-art methods, correctly assigning all jets in $93.0%$ of $6$-jet,
$87.8%$ of $7$-jet, and $82.6%$ of $\geq 8$-jet events respectively.
