  In large $N$ chaotic quantum systems, the butterfly effect is mediated by a
collective field mode known as the ``scramblon.'' We study self-interactions of
the scramblon in variants of the Sachdev-Ye-Kitaev model. In spatially extended
versions of the model and for large spatial separation, fluctuations described
by loop diagrams can invalidate the single-scramblon approximation well before
its contribution to out-of-time-order correlators becomes of order one. We find
a qualitative difference between an incoherent regime at high temperaure (or in
a Brownian version of the model) and a coherent regime at low temperature.
