  Popular guidance on observational data analysis states that outcomes should
be blinded when determining matching criteria or propensity scores. Such a
blinding is informally said to maintain the "objectivity" of the analysis, and
to prevent analysts from fishing for positive results by exploiting chance
imbalances. Contrary to this notion, we show that outcome blinding is not a
sufficient safeguard against fishing. Blinded and unblinded analysts can
produce bias of the same order of magnitude in cases where the outcomes can be
approximately predicted from baseline covariates. We illustrate this
vulnerability with a combination of analytical results and simulations.
Finally, to show that outcome blinding is not necessary to prevent bias, we
outline an alternative sample partitioning procedure for estimating the average
treatment effect on the controls, or the average treatment effect on the
treated. This procedure uses all of the the outcome data from all partitions in
the final analysis step, but does not require the analysis to not be fully
prespecified.
