  We realize a non-degenerate two-mode Dicke model with competing interactions
in a Bose-Einstein condensate (BEC) coupled to two orthogonal polarization
modes of a single optical cavity. The BEC is coupled to the cavity modes via
the scalar and vectorial part of the atomic polarizability. We can
independently change these couplings and determine their effect on a
self-organization phase transition. Measuring the phases of the system, we
characterize a crossover from a single-mode to a two-mode Dicke model. This
work provides perspectives for the realization of coupled phases of spin and
density.
