  Particle production from secondary proton-proton collisions, commonly
referred to as pile-up, impair the sensitivity of both new physics searches and
precision measurements at LHC experiments. We propose a novel algorithm, PUMA,
for identifying pile-up objects with the help of deep neural networks based on
sparse transformers. These attention mechanisms were developed for natural
language processing but have become popular in other applications. In a
realistic detector simulation, our method outperforms classical benchmark
algorithms for pile-up mitigation in key observables. It provides a perspective
for mitigating the effects of pile-up in the high luminosity era of the LHC,
where up to 200 proton-proton collisions are expected to occur simultaneously.
