  We seek to understand the probability an individual benefits from treatment
(PIBT), an inestimable quantity that must be bounded in practice. Given the
innate uncertainty in the population-level bounds on PIBT, we seek to better
understand the margin of error for their estimation in order to discern whether
the estimated bounds on PIBT are tight or wide due to random chance or not.
Toward this goal, we present guarantees to the estimation of bounds on marginal
PIBT, with any threshold of interest, for a randomized experiment setting or an
observational setting where propensity scores are known. We also derive results
that permit us to understand heterogeneity in PIBT across learnable sub-groups
delineated by pre-treatment features. These results can be used to help with
formal statistical power analyses and frequentist confidence statements for
settings where we are interested in assumption-lean inference on PIBT through
the target bounds with minimal computational complexity compared to a bootstrap
approach. Through a real data example from a large randomized experiment, we
also demonstrate how our results for PIBT can allow us to understand the
practical implication and goodness of fit of an estimate for the conditional
average treatment effect (CATE), a function of an individual's baseline
covariates.
