  We propose an algorithm for optimizations in which the gradients contain
stochastic noise. This arises, for example, in structural optimizations when
computations of forces and stresses rely on methods involving Monte Carlo
sampling, such as quantum Monte Carlo or neural network states, or are
performed on quantum devices which have intrinsic noise. Our proposed algorithm
is based on the combination of two key ingredients: an update rule derived from
the steepest descent method, and a staged scheduling of the targeted
statistical error and step-size, with position averaging. We compare it with
commonly applied algorithms, including some of the latest machine learning
optimization methods, and show that the algorithm consistently performs
efficiently and robustly under realistic conditions. Applying this algorithm,
we achieve full-degree optimizations in solids using ab initio many-body
computations, by auxiliary-field quantum Monte Carlo with planewaves and
pseudopotentials. A new metastable structure in Si was discovered in a mixed
geometry and lattice relaxing simulation. In addition to structural
optimization in materials, our algorithm can potentially be useful in other
problems in various fields where optimization with noisy gradients is needed.
