  We develop a field theory of quantum magnets and magnetic (semi)metals, which
is suitable for the analysis of their universal and topological properties. The
systems of interest include collinear, coplanar and general non-coplanar
magnets. At the basic level, we describe the dynamics of magnetic moments using
smooth vector fields in the continuum limit. Dzyaloshinskii-Moriya interaction
is captured by a non-Abelian vector gauge field, and chiral spin couplings
related to topological defects appear as higher-rank antisymmetric tensor gauge
fields. We distinguish type-I and type-II magnets by their equilibrium response
to the non-Abelian gauge flux, and characterize the resulting lattices of
skyrmions and hedgehogs, the spectra of spin waves, and the chiral response to
external perturbations. The general spin-orbit coupling of electrons is
similarly described by non-Abelian gauge fields, including higher-rank tensors
related to the electronic Berry flux. Itinerant electrons and local moments
exchange their gauge fluxes through Kondo and Hund interactions. Hence, by
utilizing gauge fields, this theory provides a unifying physical picture of
``intrinsic'' and ``topological'' anomalous Hall effects, spin-Hall effects,
and other correlations between the topological properties of electrons and
moments. We predict ``topological'' magnetoelectric effect in materials prone
to hosting hedgehogs. Links to experiments and model calculations are provided
by deriving the couplings and gauge fields from generic microscopic models,
including the Hubbard model with spin-orbit interactions. We establish the
possible existence of novel quantum spin liquids that exhibit a fractional
magnetoelectric effect, and discuss a few applications to topological magnetic
conductors like Mn$_3$Sn and Pr$_2$Ir$_2$O$_7$.
