  Hybrid classical quantum optimization methods have become an important tool
for efficiently solving problems in the current generation of NISQ computers.
These methods use an optimization algorithm executed in a classical computer,
fed with values of the objective function obtained in a quantum processor. A
proper choice of optimization algorithm is essential to achieve good
performance. Here, we review the use of first-order, second-order, and quantum
natural gradient stochastic optimization methods, which are defined in the
field of real numbers, and propose new stochastic algorithms defined in the
field of complex numbers. The performance of all methods is evaluated by means
of their application to variational quantum eigensolver, quantum control of
quantum states, and quantum state estimation. In general, complex number
optimization algorithms perform best, with first-order complex algorithms
consistently achieving the best performance, closely followed by complex
quantum natural algorithms, which do not require expensive hyperparameters
calibration. In particular, the scalar formulation of the complex quantum
natural algorithm allows to achieve good performance with low classical
computational cost.
