  The QED-C suite of Application-Oriented Benchmarks provides the ability to
gauge performance characteristics of quantum computers as applied to real-world
applications. Its benchmark programs sweep over a range of problem sizes and
inputs, capturing key performance metrics related to the quality of results,
total time of execution, and quantum gate resources consumed. In this
manuscript, we investigate challenges in broadening the relevance of this
benchmarking methodology to applications of greater complexity. First, we
introduce a method for improving landscape coverage by varying algorithm
parameters systematically, exemplifying this functionality in a new scalable
HHL linear equation solver benchmark. Second, we add a VQE implementation of a
Hydrogen Lattice simulation to the QED-C suite, and introduce a methodology for
analyzing the result quality and run-time cost trade-off. We observe a decrease
in accuracy with increased number of qubits, but only a mild increase in the
execution time. Third, unique characteristics of a supervised machine-learning
classification application are explored as a benchmark to gauge the
extensibility of the framework to new classes of application. Applying this to
a binary classification problem revealed the increase in training time required
for larger anzatz circuits, and the significant classical overhead. Fourth, we
add methods to include optimization and error mitigation in the benchmarking
workflow which allows us to: identify a favourable trade off between
approximate gate synthesis and gate noise; observe the benefits of measurement
error mitigation and a form of deterministic error mitigation algorithm; and to
contrast the improvement with the resulting time overhead. Looking ahead, we
discuss how the benchmark framework can be instrumental in facilitating the
exploration of algorithmic options and their impact on performance.
