  A key issue in science is assessing robustness to data analysis choices,
while avoiding selective reporting and providing valid inference. Specification
Curve Analysis is a tool intended to prevent selective reporting. Alas, when
used for inference it can create severe biases and false positives, due to
wrongly adjusting for covariates, and mask important treatment effect
heterogeneity. As our motivating application, it led an influential study to
conclude there is no relevant association between technology use and teenager
mental well-being. We discuss these issues and propose a strategy for valid
inference. Bayesian Specification Curve Analysis (BSCA) uses Bayesian Model
Averaging to incorporate covariates and heterogeneous effects across
treatments, outcomes and sub-populations. BSCA gives significantly different
insights into teenager well-being, revealing that the association with
technology differs by device, gender and who assesses well-being (teenagers or
their parents).
