  Gauge invariance is the cornerstone of modern quantum field theory. Recently,
it has been shown that the quantum Rabi model, describing the dipolar coupling
between a two-level atom and a quantized electromagnetic field, violates this
principle. This widely used model describes a plethora of quantum systems and
physical processes under different interaction regimes. In the ultrastrong
coupling regime, it provides predictions which drastically depend on the chosen
gauge. This failure is attributed to the finite-level truncation of the matter
system. We show that a careful application of the gauge principle is able to
restore gauge invariance even for extreme light-matter interaction regimes. The
resulting quantum Rabi Hamiltonian in the Coulomb gauge differs significantly
from the standard model and provides the same physical results obtained by
using the dipole gauge. It contains field operators to all orders that cannot
be neglected when the coupling strength is high. These results shed light on
subtleties of gauge invariance in nonperturbative and extreme interaction
regimes, which are now experimentally accessible, and solve all the
long-lasting controversies arising from gauge ambiguities in the quantum Rabi
and Dicke models.
