  A randomized trial and an analysis of observational data designed to emulate
the trial sample observations separately, but have the same eligibility
criteria, collect information on some shared baseline covariates, and compare
the effects of the same treatments on the same outcomes. Treatment effect
estimates from the trial and its emulation can be compared to benchmark
observational analysis methods. In a simplified setting with complete adherence
to the assigned treatment strategy and no loss-to-follow-up, we show that
benchmarking relies on an exchangeability condition between the populations
underlying the trial and its emulation, to account for differences in the
distribution of covariates between them. When this exchangeability condition
holds, and the usual conditions needed for the estimates from the trial and its
emulation to have a causal interpretation also hold, we derive restrictions on
the law of the observed data. When the data are compatible with the
restrictions, joint analysis of the trial and its emulation is possible. When
the data are incompatible with the restrictions, a discrepancy between (1)
estimates based on extending inferences from the trial to the population
underlying the emulation and (2) the emulation itself may reflect either
inability to benchmark (e.g., due to selective participation into the trial) or
a failure of the emulation (e.g., due to unmeasured confounding), but we cannot
use the data to determine which is the case. Our analysis reveals how
benchmarking attempts combine causal assumptions, data analysis methods, and
substantive knowledge to examine the validity of observational analysis
methods.
