  Benchmarking the performance of quantum optimization algorithms is crucial
for identifying utility for industry-relevant use cases. Benchmarking processes
vary between optimization applications and depend on user-specified goals. The
heuristic nature of quantum algorithms poses challenges, especially when
comparing to classical counterparts. A key problem in existing benchmarking
frameworks is the lack of equal effort in optimizing for the best quantum and,
respectively, classical approaches. This paper presents a comprehensive set of
guidelines comprising universal steps towards fair benchmarks. We discuss (1)
application-specific algorithm choice, ensuring every solver is provided with
the most fitting mathematical formulation of a problem; (2) the selection of
benchmark data, including hard instances and real-world samples; (3) the choice
of a suitable holistic figure of merit, like time-to-solution or solution
quality within time constraints; and (4) equitable hyperparameter training to
eliminate bias towards a particular method. The proposed guidelines are tested
across three benchmarking scenarios, utilizing the Max-Cut (MC) and Travelling
Salesperson Problem (TSP). The benchmarks employ classical mathematical
algorithms, such as Branch-and-Cut (BNC) solvers, classical heuristics, Quantum
Annealing (QA), and the Quantum Approximate Optimization Algorithm (QAOA).
