  Quantum computing systems need to be benchmarked in terms of practical tasks
they would be expected to do. Here, we propose 3 "application-motivated"
circuit classes for benchmarking: deep (relevant for state preparation in the
variational quantum eigensolver algorithm), shallow (inspired by IQP-type
circuits that might be useful for near-term quantum machine learning), and
square (inspired by the quantum volume benchmark). We quantify the performance
of a quantum computing system in running circuits from these classes using
several figures of merit, all of which require exponential classical computing
resources and a polynomial number of classical samples (bitstrings) from the
system. We study how performance varies with the compilation strategy used and
the device on which the circuit is run. Using systems made available by IBM
Quantum, we examine their performance, showing that noise-aware compilation
strategies may be beneficial, and that device connectivity and noise levels
play a crucial role in the performance of the system according to our
benchmarks.
