It is commonly asserted that no consistent, interacting quantum field theory can be constructed with fields that have spin greater than 2 (possibly with some allusion to renormalization). I've also seen (see Bailin and Love, Supersymmetry) that we cannot have helicity greater than 1, absenting gravity. I am yet to see an explanation as to why this is the case; so can anyone help?
Higher spin particles have to be coupled to conserved currents, and there are no conserved currents of high spin in quantum field theories. The only conserved currents are vector currents associated with internal symmetries, the stress-energy tensor current, the angular momentum tensor current, and the spin-3/2 supercurrent, for a supersymmetric theory.

This restriction on the currents constrains the spins to 0,1/2 (which do not need to be coupled to currents), spin 1 (which must be coupled to the vector currents), spin 3/2 (which must be coupled to a supercurrent) and spin 2 (which must be coupled to the stress-energy tensor). The argument is heuristic, and I do not think it rises to the level of a mathematical proof, but it is plausible enough to be a good guide.

Preliminaries: All possible symmetries of the S-matrix

You should accept the following result of O'Raferteigh, Coleman and Mandula--- the continuous symmetries of the particle S-matrix, assuming a mass-gap and Lorentz invariance, are a Lie Group of internal symmetries, plus the Lorentz group. This theorem is true, given its assumptions, but these assumptions leave out a lot of interesting physics:


Coleman-Mandula assume that the symmetry is a symmetry of the S-matrix, meaning that it acts nontrivially on some particle state. This seems innocuous, until you realize that you can have a symmetry which doesn't touch particle states, but only acts nontrivially on objects like strings and membranes. Such symmetries would only be relevant for the scattering of infinitely extended infinite energy objects, so it doesn't show up in the S-matrix. The transformations would become trivial whenever these sheets close in on themselves to make a localized particle. If you look at Coleman and Mandula's argument (a simple version is presented in Argyres' supersymmetry notes, which gives the flavor. There is an excellent complete presentation in Weinberg's quantum field theory book, and the original article is accessible and clear), it almost begs for the objects which are charged under the higher symmetry to be spatially extended. When you have extended fundamental objects, it is not clear that you are doing field theory anymore. If the extended objects are solitons in a renormalizable field theory, you can zoom in on ultra-short distance scattering, and consider the ultra-violet fixed point theory as the field theory you are studying, and this is sufficient to understand most examples. But the extended-object exception is the most important one, and must always be kept in the back of the mind.
Coleman and Mandula assume a mass gap. The standard extension of this theorem to the massless case just extends the maximal symmetry from the Poincare group to the conformal group, to allow the space-time part to be bigger. But Coleman and Madula use analyticity properties which I am not sure can be used in a conformal theory with all the branch-cuts which are not controlled by mass-gaps. The result is extremely plausible, but I am not sure if it is still rigorously true. This is an exercise in Weinberg, which unfortunately I haven't done.
Coleman and Mandula ignore supersymmetries. This is fixed by Haag–Lopuszanski–Sohnius, who use the Coleman mandula theorem to argue that the maximal symmetry structure of a quantum field theory is a superconformal group plus internal symmetries, and that the supersymmetry must close on the stress-energy tensor.


What the Coleman Mandula theorem means in practice is that whenever you have a conserved current in a quantum field theory, and this current acts nontrivially on particles, then it must not carry any space-time indices other than the vector index, with the only exceptions being the geometric currents: a spinor supersymmetry current, $J^{\alpha\mu}$, the (Belinfante symmetric) stress-energy tensor $T^{\mu\nu}$, the (Belinfante) angular momentum tensor $S^{\mu\nu\lambda} = x^{\mu} T^{\nu\lambda} - x^\nu T^{\mu\lambda}$, and sometimes the dilation current $D^\mu = x^\mu T^\alpha_\alpha$ and conformal and superconformal currents too.

The spin of the conserved currents is found by representation theory--- antisymmetric indices are spin 1, whether there are 1 or 2, so the spin of the internal symmetry currents is 1, and of the stress energy tensor is 2. The other geometric tensors derived from the stress energy tensor are also restricted to spin less then 2, with the supercurrent having spin 3/2.

What is a QFT?

Here this is a practical question--- for this discussion, a quantum field theory is a finite collection of local fields, each corresponding to a representation of the Poincare group, with a local interaction Lagrangian which couples them together. Further, it is assumed that there is an ultra-violet regime where all the masses are irrelevant, and where all the couplings are still relatively small, so that perturbative particle exchange is ok. I say pseudo-limit, because this isn't a real ultra-violet fixed point, which might not exist, and it does not require renormalizability, only unitarity in the regime where the theory is still perturbative.

Every particle must interact with something to be part of the theory. If you have a noninteracting sector, you throw it away as unobservable. The theory does not have to be renormalizable, but it must be unitary, so that the amplitudes must unitarize perturbatively. The couplings are assumed to be weak at some short distance scale, so that you don't make a big mess at short distances, but you can still analyze particle emission order by order

The Froissart bound for a mass-gap theory states that the scattering amplitude cannot grow faster than the logarithm of the energy. This means that any faster than constant growth in the scattering amplitude must be cancelled by something.

Propagators for any spin

The propagators for massive/massless particles of any spin follow from group theory considerations. These propagators have the schematic form

$$ s^J\over s-m^2$$

And the all-important s scaling, with its J-dependence can be extracted from the physically obvious angular dependence of the scattering amplitude. If you exchange a spin-J particle with a short propagation distance (so that the mass is unimportant) between two long plane waves (so that their angular momentum is zero), you expect the scattering amplitude to go like $\cos(\theta)^J$, just because rotations act on the helicity of the exchanged particle with this factor.

For example, when you exchange an electron between an electron and a positron, forming two photons, and the internal electron has an average momentum k and a helicity +, then if you rotate the contribution to the scattering amplitude from this exchange around the k-axis by an angle $\theta$ counterclockwise, you should get a phase of $\theta/2$ in the outgoing photon phases.

In terms of Mandelstam variables, the angular amplitude goes like $(1-t)^J$, since t is the cosine of the scattering variable, up to some scaling in s. For large t, this grows as t^J, but "t" is the "s" of a crossed channel (up to a little bit of shifting), and so crossing t and s, you expect the growth to go with the power of the angular dependence. The denominator is fixed at $J=0$, and this law is determined by Regge theory.

So that for $J=0,1/2$, the propagators shrink at large momentum, for $J=1$, the scattering amplitudes are constant in some directions, and for $J>1$ they grow. This schematic structure is of course complicated by the actual helicity states you attach on the ends of the propagator, but the schematic form is what you use in Weinberg's argument. 

Spin 0, 1/2 are OK

That spin 0 and 1/2 are ok with no special treatment, and this argument shows you why: the propagator for spin 0 is

$$ 1\over k^2 + m^2$$

Which falls off in k-space at large k. This means that when you scatter by exchanging scalars, your tree diagrams are shrinking, so that they don't require new states to make the theory unitary.

Spinors have a propagator

$$ 1\over \gamma\cdot k + m $$

This also falls off at large k, but only linearly. The exchange of spinors does not make things worse, because spinor loops tend to cancel the linear divergence by symmetry in k-space, leaving log divergences which are symptomatic of a renormalizable theory.

So spinors and scalars can interact without revealing substructure, because their propagators do not require new things for unitarization. This is reflected in the fact that they can make renormalizable theories all by themselves.

Spin 1

Introducing spin 1, you get a propagator that doesn't fall off. The massive propagator for spin 1 is

$$ { g_{\mu\nu} - {k_\mu k_\nu\over m^2} \over k^2 + m^2 }$$

The numerator projects the helicity to be perpendicular to k, and the second term is problematic. There are directions in k-space where the propagator does not fall off at all! This means that when you scatter by spin-1 exchange, these directions can lead to a blow-up in the scattering amplitude at high energies which has to be cancelled somehow.

If you cancel the divergence with higher spin, you get a divergence there, and you need to cancel that, and then higher spin, and so on, and you get infinitely many particle types. So the assumption is that you must get rid of this divergence intrinsically. The way to do this is to assume that the $k_\mu k_\nu$ term is always hitting a conserved current. Then it's contribution vanishes.

This is what happens in massive electrodynamics. In this situation, the massive propagator is still ok for renormalizability, as noted by Schwinger and Feynman, and explained by Stueckelberg. The $k_\mu k_\nu$ is always hitting a $J^\mu$, and in x-space, it is proportional to the divergence of the current, which is zero because the current is conserved even with a massive photon (because the photon isn't charged).

The same argument works to kill the k-k part of the propagator in Yang-Mills fields, but it is much more complicated, because the Yang-Mills field itself is charged, so the local conservation law is usually expressed in a different way, etc,etc. The heuristic lesson is that spin-1 is only ok if you have a conservation law which cancels the non-shrinking part of the numerator. This requires Yang-Mills theory, and the result is also compatible with renormalizability.

If you have a spin-1 particle which is not a Yang-Mills field, you will need to reveal new structure to unitarize its longitudinal component, whose propagator is not properly shrinking at high energies.

Spin 3/2

In this case, you have a Rarita Schwinger field, and the propagator is going to grow like $\sqrt{s}$ at large energies, just from the Mandelstam argument presented before.

The propagator growth leads to unphysical growth in scattering exchanging this particle, unless the spin-3/2 field is coupled to a conserved current. The conserved current is the Supersymmetry current, by the Haag–Lopuszanski–Sohnius theorem, because it is a spinor of conserved currents.

This means that the spin-3/2 particle should interact with a spin 3/2 conserved supercurrent in order to be consistent, and the number of gravitinos is (less then or equal to) the number of supercharges.

The gravitinos are always introduced in a supermultiplet with the graviton, but I don't know if it is definitely impossible to introduce them with a spin-1 partner, and couple them to the supercurrent anyway. These spin-3/2/spin-1 multiplets will probably not be renormalizable barring some supersymmetry miracle. I haven't worked it out, but it might be possible.

Spin 2

In this case, you have a perturbative graviton-like field $h_{\mu\nu}$, and the propagator contains terms growing linearly with s.

In order to cancel the growth in the numerator, you need the tensor particle to be coupled to a conserved current to kill the parts with too-rapid growth, and produce a theory which does not require new particles for unitarity. The conserved quantity must be a tensor $T_{\mu\nu}$. Now one can appeal to the Coleman Mandula theorem and conclude that the conserved tensor current must be the stress energy tensor, and this gives general relativity, since the stress-tensor includes the stress of the h field too.

There is a second tensor conserved quantity, the angular momentum tensor $S_{\mu\nu\sigma}$, which is also spin-2 (it might look like its spin 3, but its antisymmetric on two of its indices). You can try to couple a spin-2 field to the angular momentum tensor. To see if this works requires a detailed analysis, which I haven't done, but I would guess that the result will just be a non-dynamical torsion coupled to the local spin, as required by the Einstein-Cartan theory.

Witten mentions yet another possiblity for spin 2 in chapter 1 of Green Schwarz and Witten, but I don't remember what it is, and I don't know whether it is viable.

Summary

I believe that these arguments are due to Weinberg, but I personally only read the sketchy summary of them in the first chapters of Green Schwarz and Witten. They do not seem to me to have the status of a theorem, because the argument is particle by particle, it requires independent exchange in a given regime, and it discounts the possiblity that unitary can be restored by some family of particles.

Of course, in string theory, there are fields of arbitrarily high spin, and unitarity is restored by propagating all of them together. For field theories with bound states which lie on Regge trajectories, you can have arbitrarily high spins too, so long as you consider all the trajectory contributions together, to restore unitarity (this was one of the original motivations for Regge theory--- unitarizing higher spin theories).

For example, in QCD, we have nuclei of high ground-state spin. So there are stable S-matrix states of high spin, but they come in families with other excited states of the same nuclei.

The conclusion here is that if you have higher spin particles, you can be pretty sure that you will have new particles of even higher spin at higher energies, and this chain of particles will not stop until you reveal new structure at some point. So the tensor mesons observed in the strong interaction mean that you should expect an infinite family of strongly interacting particles, petering out only when the quantum field substructure is revealed.

Some comments

James said:


It seems higher spin fields must be massless so that they have a gauge symmetry and thus a current to couple to
A massless spin-2 particle can only be a graviton.


These statements are as true as the arguments above are convincing. From the cancellation required for the propagator to become sensible, higher spin fields are fundamentally massless at short distances. The spin-1 fields become massive by the Higgs mechanism, the spin 3/2 gravitinos become massive through spontaneous SUSY breaking, and this gets rid of Goldstone bosons/Goldstinos.

But all this stuff is, at best, only at the "mildly plausible" level of argument--- the argument is over propagator unitarization with each propagator separately having no cancellations. It's actually remarkable that it works as a guideline, and that there aren't a slew of supersymmetric exceptions of higher spin theories with supersymmetry enforcing propagator cancellations and unitarization. Maybe there are, and they just haven't been discovered yet. Maybe there's a better way to state the argument which shows that unitarity can't be restored by using positive spectral-weight particles.

Big Rift in 1960s

James askes 


Why wasn't this pointed out earlier in the history of string theory?


The history of physics cannot be well understood without appreciating the unbelievable antagonism between the Chew/Mandelstam/Gribov S-matrix camp, and the Weinberg/Glashow/Polyakov Field theory camp. The two sides hated each other, did not hire each other, and did not read each other, at least not in the west. The only people that straddled both camps were older folks and Russians--- Gell-Mann more than Landau (who believed the Landau pole implied S-matrix), Gribov and Migdal more than anyone else in the west other than Gell-Mann and Wilson. Wilson did his PhD in S-matrix theory, for example, as did David Gross (under Chew).

In the 1970s, S-matrix theory just plain died. All practitioners jumped ship rapidly in 1974, with the triple-whammy of Wilsonian field theory, the discovery of the Charm quark, and asymptotically freedom. These results killed S-matrix theory for thirty years. Those that jumped ship include all the original string theorists who stayed employed: notably Veneziano, who was convinced that gauge theory was right when t'Hooft showed that large-N gauge fields give the string topological expansion, and Susskind, who didn't mention Regge theory after the early 1970s. Everybody stopped studying string theory except Scherk and Schwarz, and Schwarz was protected by Gell-Mann, or else he would never have been tenured and funded.

This sorry history means that not a single S-matrix theory course is taught in the curriculum today, nobody studies it except a few theorists of advanced age hidden away in particle accelerators, and the main S-matrix theory, string-theory, is not properly explained and remains completely enigmatic even to most physicists. There were some good reasons for this--- some S-matrix people said silly things about the consistency of quantum field theory--- but to be fair, quantum field theory people said equally silly things about S-matrix theory.

Weinberg came up with these heuristic arguments in the 1960s, which convinced him that S-matrix theory was a dead end, or rather, to show that it was a tautological synonym for quantum field theory. Weinberg was motivated by models of pion-nucleon interactions, which was a hot S-matrix topic in the early 1960s. The solution to the problem is the chiral symmetry breaking models of the pion condensate, and these are effective field theories.

Building on this result, Weinberg became convinced that the only real solution to the S-matrix was a field theory of some particles with spin. He still says this every once in a while, but it is dead wrong. The most charitable interpretation is that every S-matrix has a field theory limit, where all but a finite number of particles decouple, but this is not true either (consider little string theory). String theory exists, and there are non-field theoretic S-matrices, namely all the ones in string theory, including little string theory in (5+1)d, which is non-gravitational.

Lorentz indices

James comments: 


regarding spin, I tried doing the group theoretic approach to an antisymmetric tensor but got a little lost - doesn't an antisymmetric 2-form (for example) contain two spin-1 fields?


The group theory for an antisymmetric tensor is simple: it consists of an "E" and "B" field which can be turned into the pure chiral representations E+iB, E-iB. This was also called a "six-vector" sometimes, meaning E,B making an antisymmetric four-tensor.

You can do this using dotted and undotted indices more easily, if you realize that the representation theory of SU(2) is best done in indices--- see the "warm up" problem in this answer: Mathematically, what is color charge?
