  We propose a very large family of benchmarks for probing the performance of
quantum computers. We call them volumetric benchmarks (VBs) because they
generalize IBM's benchmark for measuring quantum volume \cite{Cross18}. The
quantum volume benchmark defines a family of square circuits whose depth $d$
and width $w$ are the same. A volumetric benchmark defines a family of
rectangular quantum circuits, for which $d$ and $w$ are uncoupled to allow the
study of time/space performance trade-offs. Each VB defines a mapping from
circuit shapes -- $(w,d)$ pairs -- to test suites $\mathcal{C}(w,d)$. A test
suite is an ensemble of test circuits that share a common structure. The test
suite $\mathcal{C}$ for a given circuit shape may be a single circuit $C$, a
specific list of circuits $\{C_1\ldots C_N\}$ that must all be run, or a large
set of possible circuits equipped with a distribution $Pr(C)$. The circuits in
a given VB share a structure, which is limited only by designers' creativity.
We list some known benchmarks, and other circuit families, that fit into the VB
framework: several families of random circuits, periodic circuits, and
algorithm-inspired circuits. The last ingredient defining a benchmark is a
success criterion that defines when a processor is judged to have "passed" a
given test circuit. We discuss several options. Benchmark data can be analyzed
in many ways to extract many properties, but we propose a simple, universal
graphical summary of results that illustrates the Pareto frontier of the $d$ vs
$w$ trade-off for the processor being benchmarked.
  [1] A. Cross, et al., Phys. Rev. A, 100, 032328, September 2019.
