  We propose an efficient method for simultaneously optimizing both the
structure and parameter values of quantum circuits with only a small
computational overhead. Shallow circuits that use structure optimization
perform significantly better than circuits that use parameter updates alone,
making this method particularly suitable for noisy intermediate-scale quantum
computers. We demonstrate the method for optimizing a variational quantum
eigensolver for finding the ground states of Lithium Hydride and the Heisenberg
model in simulation, and for finding the ground state of Hydrogen gas on the
IBM Melbourne quantum computer.
