  Certain patterns of symmetry fractionalization in topologically ordered
phases of matter are anomalous, in the sense that they can only occur at the
surface of a higher dimensional symmetry-protected topological (SPT) state. An
important question is to determine how to compute this anomaly, which means
determining which SPT hosts a given symmetry-enriched topological order at its
surface. While special cases are known, a general method to compute the anomaly
has so far been lacking. In this paper we propose a general method to compute
relative anomalies between different symmetry fractionalization classes of a
given (2+1)D topological order. This method applies to all types of symmetry
actions, including anyon-permuting symmetries and general space-time reflection
symmetries. We demonstrate compatibility of the relative anomaly formula with
previous results for diagnosing anomalies for $\mathbb{Z}_2^{\bf T}$ space-time
reflection symmetry (e.g. where time-reversal squares to the identity) and
mixed anomalies for $U(1) \times \mathbb{Z}_2^{\bf T}$ and $U(1) \rtimes
\mathbb{Z}_2^{\bf T}$ symmetries. We also study a number of additional
examples, including cases where space-time reflection symmetries are
intertwined in non-trivial ways with unitary symmetries, such as
$\mathbb{Z}_4^{\bf T}$ and mixed anomalies for $\mathbb{Z}_2 \times
\mathbb{Z}_2^{\bf T}$ symmetry, and unitary $\mathbb{Z}_2 \times \mathbb{Z}_2$
symmetry with non-trivial anyon permutations.
