  We introduce quantum utility, a new approach to evaluating quantum
performance that aims to capture the user experience by including overhead
costs associated with the quantum computation. A demonstration of quantum
utility by a quantum processing unit (QPU) shows that the QPU can outperform
classical solvers at some tasks of interest to practitioners, when considering
computational overheads. We consider overhead costs that arise in standalone
use of the QPU (as opposed to a hybrid computation context). We define three
early milestones on the path to broad-scale quantum utility that focus on
restricted subsets of overheads: Milestone 0 considers pure anneal time (no
overheads) and has been demonstrated in previous work; Milestone 1 includes
overhead times to access the QPU (that is, programming and readout); and
Milestone 2 incorporates an indirect cost associated with minor embedding.
  We evaluate the performance of a D-Wave Advantage QPU with respect to
Milestones 1 and 2, using a testbed of 13 input classes and seven classical
solvers implemented on CPUs and GPUs. For Milestone 1, the QPU outperformed all
classical solvers in 99% of our tests. For Milestone 2, the QPU outperformed
all classical solvers in 19% of our tests, and the scenarios in which the QPU
found success correspond to cases where classical solvers most frequently
failed.
  Analysis of test results on specific inputs reveals fundamentally distinct
underlying mechanisms that explain the observed differences in quantum and
classical performance profiles. We present evidence-based arguments that these
distinctions bode well for future annealing quantum processors to support
demonstrations of quantum utility on ever-expanding classes of inputs and for
more challenging milestones.
