  The detrimental effect of noise accumulates as quantum computers grow in
size. In the case where devices are too small or noisy to perform error
correction, error mitigation may be used. Error mitigation does not increase
the fidelity of quantum states, but instead aims to reduce the approximation
error in quantities of concern, such as expectation values of observables.
However, it is as yet unclear which circuit types, and devices of which
characteristics, benefit most from the use of error mitigation. Here we develop
a methodology to assess the performance of quantum error mitigation techniques.
Our benchmarks are volumetric in design, and are performed on different
superconducting hardware devices. Extensive classical simulations are also used
for comparison. We use these benchmarks to identify disconnects between the
predicted and practical performance of error mitigation protocols, and to
identify the situations in which their use is beneficial. To perform these
experiments, and for the benefit of the wider community, we introduce Qermit -
an open source python package for quantum error mitigation. Qermit supports a
wide range of error mitigation methods, is easily extensible and has a modular
graph-based software design that facilitates composition of error mitigation
protocols and subroutines.
