  Many recent developments in causal inference, and functional estimation
problems more generally, have been motivated by the fact that classical
one-step (first-order) debiasing methods, or their more recent sample-split
double machine-learning avatars, can outperform plugin estimators under
surprisingly weak conditions. These first-order corrections improve on plugin
estimators in a black-box fashion, and consequently are often used in
conjunction with powerful off-the-shelf estimation methods. These first-order
methods are however provably suboptimal in a minimax sense for functional
estimation when the nuisance functions live in Holder-type function spaces.
This suboptimality of first-order debiasing has motivated the development of
"higher-order" debiasing methods. The resulting estimators are, in some cases,
provably optimal over Holder-type spaces, but both the estimators which are
minimax-optimal and their analyses are crucially tied to properties of the
underlying function space.
  In this paper we investigate the fundamental limits of structure-agnostic
functional estimation, where relatively weak conditions are placed on the
underlying nuisance functions. We show that there is a strong sense in which
existing first-order methods are optimal. We achieve this goal by providing a
formalization of the problem of functional estimation with black-box nuisance
function estimates, and deriving minimax lower bounds for this problem. Our
results highlight some clear tradeoffs in functional estimation -- if we wish
to remain agnostic to the underlying nuisance function spaces, impose only
high-level rate conditions, and maintain compatibility with black-box nuisance
estimators then first-order methods are optimal. When we have an understanding
of the structure of the underlying nuisance functions then carefully
constructed higher-order estimators can outperform first-order estimators.
