  We study the non-equilibrium phase diagram and the dynamical phase
transitions occurring during the pre-thermalization of non-integrable quantum
spin chains, subject to either quantum quenches or linear ramps of a relevant
control parameter. We consider spin systems in which long-range ferromagnetic
interactions compete with short-range, integrability-breaking terms. We capture
the pre-thermal stages of the non-equilibrium evolution via a time-dependent
spin wave expansion at leading order in the spin waves density. In order to
access regimes with strong integrability breaking, instead, we perform
numerical simulations based on the time-dependent variational principle with
matrix product states. By investigating a large class of quantum spin models,
we demonstrate that non-equilibrium fluctuations can significantly affect the
dynamics near critical points of the phase diagram, resulting in a chaotic
evolution of the collective order parameter, akin to the dynamics of a
classical particle in a multiple-well potential subject to quantum friction. We
also elucidate the signature of this novel dynamical phase on the
time-dependent correlation functions of the local order parameter. We finally
establish a connection with the notion of dynamical quantum phase transition
associated with a possible non-analytic behavior of the return probability
amplitude, or Loschmidt echo, showing that the latter displays cusps whenever
the order parameter vanishes during its real-time evolution.
