  Learned systems in the domain of visual recognition and cognition impress in
part because even though they are trained with datasets many orders of
magnitude smaller than the full population of possible images, they exhibit
sufficient generalization to be applicable to new and previously unseen data.
Since training data sets typically represent small sampling of a domain, the
possibility of bias in their composition is very real. But what are the limits
of generalization given such bias, and up to what point might it be sufficient
for a real problem task? Although many have examined issues regarding
generalization, this question may require examining the data itself. Here, we
focus on the characteristics of the training data that may play a role. Other
disciplines have grappled with these problems, most interestingly epidemiology,
where experimental bias is a critical concern. The range and nature of data
biases seen clinically are really quite relatable to learned vision systems.
One obvious way to deal with bias is to ensure a large enough training set, but
this might be infeasible for many domains. Another approach might be to perform
a statistical analysis of the actual training set, to determine if all aspects
of the domain are fairly captured. This too is difficult, in part because the
full set of variables might not be known, or perhaps not even knowable. Here,
we try a different approach in the tradition of the Thought Experiment, whose
most famous instance may be Schr\"odinger's Cat. There are many types of bias
as will be seen, but we focus only on one, selection bias. The point of the
thought experiment is not to demonstrate problems with all learned systems.
Rather, this might be a simple theoretical tool to probe into bias during data
collection to highlight deficiencies that might then deserve extra attention
either in data collection or system development.
