  Scientific experiments study interventions that show evidence of an effect
size that is meaningfully large, negligibly small, or inconclusively broad.
Previously, we proposed contra-analysis as a decision-making process to help
determine which interventions have a meaningfully large effect by using contra
plots to compare effect size across broadly related experiments. Here, we
extend the use of contra plots to determine which results have evidence of
negligible (near-zero) effect size. Determining if an effect size is negligible
is important for eliminating alternative scientific explanations and
identifying approximate independence between an intervention and the variable
measured. We illustrate that contra plots can score negligible effect size
across studies, inform the selection of a threshold for negligible effect based
on broadly related results, and determine which results have evidence of
negligible effect with a hypothesis test. No other data visualization can carry
out all three of these tasks for analyzing negligible effect size. We
demonstrate this analysis technique on real data from biomedical research. This
new application of contra plots can differentiate statistically insignificant
results with high strength (narrow and near-zero interval estimate of effect
size) from those with low strength (broad interval estimate of effect size).
Such a designation could help resolve the File Drawer problem in science, where
statistically insignificant results are underreported because their
interpretation is ambiguous and nonstandard. With our proposed procedure,
results designated with negligible effect will be considered strong and
publishable evidence of near-zero effect size.
