  Many near-term quantum computing algorithms are conceived as variational
quantum algorithms, in which parameterized quantum circuits are optimized in a
hybrid quantum-classical setup. Examples are variational quantum eigensolvers,
quantum approximate optimization algorithms as well as various algorithms in
the context of quantum-assisted machine learning. A common bottleneck of any
such algorithm is constituted by the optimization of the variational
parameters. A popular set of optimization methods work on the estimate of the
gradient, obtained by means of circuit evaluations. We will refer to the way in
which one can combine these circuit evaluations as gradient rules. This work
provides a comprehensive picture of the family of gradient rules that vary
parameters of quantum gates individually. The most prominent known members of
this family are the parameter shift rule and the finite differences method. To
unite this family, we propose a generalized parameter shift rule that expresses
all members of the aforementioned family as special cases, and discuss how all
of these can be seen as providing access to a linear combination of exact
first- and second-order derivatives. We further prove that a parameter shift
rule with one non-shifted evaluation and only one shifted circuit evaluation
can not exist does not exist, and introduce a novel perspective for approaching
new gradient rules.
