  We develop a new fermionic path-integral formalism to analyze the phase
diagram of open nonequilibrium systems. The formalism is applied to analyze an
ensemble of two-level atoms interacting with a single-mode optical cavity,
described by the Dicke model. While this model is often used as the
paradigmatic example of a phase transition in driven-dissipative systems,
earlier theoretical studies were limited to the special case when the total
spin of the atomic ensemble is conserved. This assumption is not justified in
most experimental realizations. Our new approach allows us to analyze the
problem in a more general case, including the experimentally relevant case of
dissipative processes that act on each atom individually and do not conserve
the total spin. We obtain a general expression for the position of the
transition, which contains as special cases the two previously known regimes:
i) non-equilibrium systems with losses and conserved spin and ii) closed
systems in thermal equilibrium and with the Gibbs ensemble averaging over the
values of the total spin. We perform a detailed study of different types of
baths and point out the possibility of a surprising non-monotonous dependence
of the transition on the baths' parameters.
