  Sign-changing interactions constitute a crucial ingredient in the creation of
frustrated many-body systems such as spin glasses. We present here the
demonstration of a photon-mediated sign-changing interaction between
Bose-Einstein condensed (BEC) atoms in a confocal cavity. The interaction
between two atoms is of an unusual, nonlocal form proportional to the cosine of
the inner product of the atoms' position vectors. This interaction arises from
the differing Gouy phase shifts of the cavity's degenerate modes. Moreover,
these Gouy phase anomalies induce an extra pattern of Z_2-symmetry-breaking in
the atomic density-wave self-ordering that arises from a nonequilibrium
Dicke-type phase transition in the system. This state is detected via the
holographic imaging of the cavity's superradiant emission. Together with Ref.
[1], we explore this interaction's influence on superradiant phase transitions
in multimode cavities. Employing this interaction in cavity QED spin systems
may enable the creation of artificial spin glasses and quantum neural networks.
