  Infrared effects in the scattering of particles in gravity and
electrodynamics entail an exchange of relativistic angular momentum between
pairs of particles and the gauge field. Due to this exchange particles can
carry an asymptotically non-vanishing "pairwise" boost-like angular momentum
proportional to the product of their couplings to the field. At the quantum
level this asymptotic angular momentum suggests the existence of a new quantum
number carried by multi-particle states. We argue that such quantum number is
related to a modification of the action of the generators of Lorentz
transformations on multi-particle states. We derive such a modification using a
group-theoretic argument based on the little group of the conformal primary
basis for asymptotic states. The corresponding representation is an extension
of the ordinary multi-particle Fock representation of the Poincar\'e group. The
new multi-particle states belonging to such representation no longer factorize
into tensor products of one-particle states. Viewed from a gravitational point
of view, our results provide evidence for a universal breakdown of the
description of multi-particle sates in terms of Fock space due to infrared
back-reaction.
