  We analyze the strong-coupling dynamics of a driven harmonic oscillator whose
energy is modulated by a continuum of other bosonic modes. This type of
system-bath interaction appears, for example, in optomechanical or equivalent
circuit QED setups, where the frequency of a confined photonic mode depends
linearly on a fluctuating boundary. Compared to the canonical spin-boson model,
where coupling to bath modes only leads to decoherence, the role of the
environment in such systems is more complex, since it also provides the only
source of nonlinearity. We show that even for an unstructured bath, these
environment-induced nonlinearities can dominate over decoherence processes
resulting in Rabi oscillations and the formation of highly non-classical
states. These findings provide important insights into the non-Markovian
dynamics of higher-dimensional open quantum systems and for realizing
few-photon optical nonlinearities through strong interactions with a bath.
