  The parameter derivative of the expectation value of the energy, $\partial
E/\partial p$, is a key ingredient in variational quantum Monte Carlo (VMC)
wave function optimization methods. In some cases, a na\"ive Monte Carlo
estimate of this derivative suffers from an infinite variance which inhibits
the efficiency of optimization methods that rely on a stable estimate of the
derivative. In this work, we derive a simple regularization of the na\"ive
estimator which is trivial to implement in existing VMC codes, has finite
variance, and a negligible bias which can be extrapolated to zero bias with no
extra cost. We use this estimator to construct an unbiased, finite variance
estimation of $\partial E/\partial p$ for a multi-Slater-Jastrow trial wave
function on the LiH molecule. This regularized estimator is a simple and
efficient estimator of $\partial E/\partial p$ for VMC optimization techniques.
