  We analyze how gauge fixing, which is required by any practical continuum
approach to gauge systems, can interfere with the physical symmetries of such
systems. In principle, the gauge fixing procedure, which deals with the
(unphysical) gauge symmetry, should not interfere with the other (physical)
symmetries. In practice, however, there can be an interference which takes two
different forms. First, depending on the considered gauge, it might not always
be simple or possible to devise approximation schemes that preserve the
physical symmetry constraints on (gauge-independent) observables. Second, even
at an exact level of discussion, the (gauge-dependent) effective action for the
gauge field, and thus the related vertex functions, may not reflect the
physical symmetries of the problem. We illustrate these difficulties using a
very general class of gauge fixings that contains the usual gauge fixings as
particular cases. Using background field techniques, we then propose specific
gauge choices that allow one to keep the physical symmetries explicit, both at
the level of the observables and at the level of the effective action for the
gauge field. Our analysis is based on the notion of invariance modulo gauge
transformations. This is not only a convenient framework to discuss symmetries
in the presence of unphysical degrees of freedom, but it also allows one to
reinterpret certain well known phenomena in gauge theories without the need to
invoke the conceptually annoying ``breaking of a gauge symmetry''.
