  The need to evaluate treatment effectiveness is ubiquitous in most of
empirical science, and interest in flexibly investigating effect heterogeneity
is growing rapidly. To do so, a multitude of model-agnostic, nonparametric
meta-learners have been proposed in recent years. Such learners decompose the
treatment effect estimation problem into separate sub-problems, each solvable
using standard supervised learning methods. Choosing between different
meta-learners in a data-driven manner is difficult, as it requires access to
counterfactual information. Therefore, with the ultimate goal of building
better understanding of the conditions under which some learners can be
expected to perform better than others a priori, we theoretically analyze four
broad meta-learning strategies which rely on plug-in estimation and
pseudo-outcome regression. We highlight how this theoretical reasoning can be
used to guide principled algorithm design and translate our analyses into
practice by considering a variety of neural network architectures as
base-learners for the discussed meta-learning strategies. In a simulation
study, we showcase the relative strengths of the learners under different
data-generating processes.
