  A screening experiment attempts to identify a subset of important effects
using a relatively small number of experimental runs. Given the limited run
size and a large number of possible effects, penalized regression is a popular
tool used to analyze screening designs. In particular, an automated
implementation of the Gauss-Dantzig selector has been widely recommended to
compare screening design construction methods. Here, we illustrate potential
reproducibility issues that arise when comparing screening designs via
simulation, and recommend a graphical method, based on screening probabilities,
which compares designs by evaluating them along the penalized regression
solution path. This method can be implemented using simulation, or, in the case
of lasso, by using exact local lasso sign recovery probabilities. Our approach
circumvents the need to specify tuning parameters associated with
regularization methods, leading to more reliable design comparisons. This
article contains supplementary materials including code to implement the
proposed methods.
