  We present a general class of unbiased improved estimators for physical
observables in lattice gauge theory computations which significantly reduces
statistical errors at modest computational cost. The error reduction
techniques, referred to as covariant approximation averaging, utilize
approximations which are covariant under lattice symmetry transformations. We
observed cost reductions from the new method compared to the traditional one,
for fixed statistical error, of 16 times for the nucleon mass at $M_\pi\sim
330$ MeV (Domain-Wall quark) and 2.6-20 times for the hadronic vacuum
polarization at $M_\pi\sim 480$ MeV (Asqtad quark). These cost reductions
should improve with decreasing quark mass and increasing lattice sizes.
