  Stimulated Brillouin scattering (SBS) is an important nonlinear optical
effect which can both enable and impede optical processes in guided wave
systems. Highly multi-mode excitation of fibers has been proposed as a novel
route towards efficient suppression of SBS in both active and passive fibers.
To study the effects of multimode excitation generally, we develop a theory of
SBS for arbitrary input excitations, fiber cross section geometries and
refractive index profiles. We derive appropriate nonlinear coupled mode
equations for the signal and Stokes modal amplitudes starting from vector
optical and tensor acoustic equations. Using applicable approximations, we find
an analytical formula for the SBS (Stokes) gain susceptibility, which takes
into account the vector nature of both optical and acoustic modes exactly. We
show that upon multimode excitation, the SBS power in each Stokes mode grows
exponentially with a growth rate that depends parametrically on the
distribution of power in the signal modes. Specializing to isotropic fibers we
are able to define and calculate an effective SBS gain spectrum for any choice
of multimode excitation. The peak value of this gain spectrum determines the
SBS threshold, the maximum SBS-limited power that can be sent through the
fiber. We show theoretically that peak SBS gain is greatly reduced by highly
multimode excitation due to gain broadening and relatively weaker intermodal
SBS gain. We demonstrate that equal excitation of the 160 modes of a
commercially available, highly multimode circular step index fiber raises the
SBS threshold by a factor of 6.5, and find comparable suppression of SBS in
similar fibers with a D-shaped cross-section.
