  Magnetism in transition-metal compounds (TMCs) has traditionally been
associated with spin degrees of freedom, because the orbital magnetic moments
are typically largely quenched. On the other hand, magnetic order in 4f- and
5d-electron systems arises from spin and orbital moments that are rigidly tied
together by the large intra-atomic spin-orbit coupling (SOC). Using inelastic
neutron scattering on the archetypal 4d-electron Mott insulator Ca$_2$RuO$_4$,
we report a novel form of excitonic magnetism in the intermediate-strength
regime of the SOC. The magnetic order is characterized by ``soft'' magnetic
moments with large amplitude fluctuations manifested by an intense, low-energy
excitonic mode analogous to the Higgs mode in particle physics. This mode
heralds a proximate quantum critical point separating the soft magnetic order
driven by the superexchange interaction from a quantum-paramagnetic state
driven by the SOC. We further show that this quantum critical point can be
tuned by lattice distortions, and hence may be accessible in epitaxial
thin-film structures. The unconventional spin-orbital-lattice dynamics in
Ca$_2$RuO$_4$ identifies the SOC as a novel source of quantum criticality in
TMCs.
