  Graphics play a crucial role in statistical analysis and data mining. This
paper describes metrics developed to assist the use of lineups for making
inferential statements. Lineups embed the plot of the data among a set of null
plots, and engage a human observer to select the plot that is most different
from the rest. If the data plot is selected it corresponds to the rejection of
a null hypothesis. Metrics are calculated in association with lineups, to
measure the quality of the lineup, and help to understand what people see in
the data plots. The null plots represent a finite sample from a null
distribution, and the selected sample potentially affects the ease or
difficulty of a lineup. Distance metrics are designed to describe how close the
true data plot is to the null plots, and how close the null plots are to each
other. The distribution of the distance metrics is studied to learn how well
this matches to what people detect in the plots, the effect of null generating
mechanism and plot choices for particular tasks. The analysis was conducted on
data that has already been collected from Amazon Turk studies conducted with
lineups for studying an array of data analysis tasks.
