  New results suggest strong limits to the feasibility of classifying human
brain activity evoked from image stimuli, as measured through EEG. Considerable
prior work suffers from a confound between the stimulus class and the time
since the start of the experiment. A prior attempt to avoid this confound using
randomized trials was unable to achieve results above chance in a statistically
significant fashion when the data sets were of the same size as the original
experiments. Here, we again attempt to replicate these experiments with
randomized trials on a far larger (20x) dataset of 1,000 stimulus presentations
of each of forty classes, all from a single subject. To our knowledge, this is
the largest such EEG data collection effort from a single subject and is at the
bounds of feasibility. We obtain classification accuracy that is marginally
above chance and above chance in a statistically significant fashion, and
further assess how accuracy depends on the classifier used, the amount of
training data used, and the number of classes. Reaching the limits of data
collection without substantial improvement in classification accuracy suggests
limits to the feasibility of this enterprise.
