  Researchers frequently test and improve model fit by holding a sample
constant and varying the model. We propose methods to test and improve sample
fit by holding a model constant and varying the sample. Much as the bootstrap
is a well-known method to re-sample data and estimate the uncertainty of the
fit of parameters in a model, we develop Sample Fit Reliability (SFR) as a set
of computational methods to re-sample data and estimate the reliability of the
fit of observations in a sample. SFR uses Scoring to assess the reliability of
each observation in a sample, Annealing to check the sensitivity of results to
removing unreliable data, and Fitting to re-weight observations for more robust
analysis. We provide simulation evidence to demonstrate the advantages of using
SFR, and we replicate three empirical studies with treatment effects to
illustrate how SFR reveals new insights about each study.
