  We study regression adjustment with general function class approximations for
estimating the average treatment effect in the design-based setting. Standard
regression adjustment involves bias due to sample re-use, and this bias leads
to behavior that is sub-optimal in the sample size, and/or imposes restrictive
assumptions. Our main contribution is to introduce a novel decorrelation-based
approach that circumvents these issues. We prove guarantees, both asymptotic
and non-asymptotic, relative to the oracle functions that are targeted by a
given regression adjustment procedure. We illustrate our method by applying it
to various high-dimensional and non-parametric problems, exhibiting improved
sample complexity and weakened assumptions relative to known approaches.
