  We discuss the nature of symmetry breaking and the associated collective
excitations for a system of bosons coupled to the electromagnetic field of two
optical cavities. For the specific configuration realized in a recent
experiment at ETH, we show that, in absence of direct intercavity scattering
and for parameters chosen such that the atoms couple symmetrically to both
cavities, the system possesses an approximate $U(1)$ symmetry which holds
asymptotically for vanishing cavity field intensity. It corresponds to the
invariance with respect to redistributing the total intensity $I=I_1+I_2$
between the two cavities. The spontaneous breaking of this symmetry gives rise
to a broken continuous translation-invariance for the atoms, creating a
supersolid-like order in the presence of a Bose-Einstein condensate. In
particular, we show that atom-mediated scattering between the two cavities,
which favors the state with equal light intensities $I_1=I_2$ and reduces the
symmetry to $\mathbf{Z}_2\otimes \mathbf{Z}_2$, gives rise to a finite value
$\sim \sqrt{I}$ of the effective Goldstone mass. For strong atom driving, this
low energy mode is clearly separated from an effective Higgs excitation
associated with changes of the total intensity $I$. In addition, we compute the
spectral distribution of the cavity light field and show that both the Higgs
and Goldstone mode acquire a finite lifetime due to Landau damping at non-zero
temperature.
