  Effective methods for characterizing the noise in quantum computing devices
are essential for programming and debugging circuit performance. Existing
approaches vary in the information obtained as well as the amount of quantum
and classical resources required, with more information generally requiring
more resources. Here we benchmark the characterization methods of gate set
tomography, Pauli channel noise reconstruction, and empirical direct
characterization for developing models that describe noisy quantum circuit
performance on a 27-qubit superconducting transmon device. We evaluate these
models by comparing the accuracy of noisy circuit simulations with the
corresponding experimental observations. We find that the agreement of noise
model to experiment does not correlate with the information gained by
characterization and that the underlying circuit strongly influences the best
choice of characterization approach. Empirical direct characterization scales
best of the methods we tested and produced the most accurate characterizations
across our benchmarks.
