  In deep learning, achieving high performance on image classification tasks
requires diverse training sets. However, the current best
practice$\unicode{x2013}$maximizing dataset size and class
balance$\unicode{x2013}$does not guarantee dataset diversity. We hypothesized
that, for a given model architecture, model performance can be improved by
maximizing diversity more directly. To test this hypothesis, we introduce a
comprehensive framework of diversity measures from ecology that generalizes
familiar quantities like Shannon entropy by accounting for similarities among
images. (Size and class balance emerge as special cases.) Analyzing thousands
of subsets from seven medical datasets showed that the best correlates of
performance were not size or class balance but $A$$\unicode{x2013}$"big
alpha"$\unicode{x2013}$a set of generalized entropy measures interpreted as the
effective number of image-class pairs in the dataset, after accounting for
image similarities. One of these, $A_0$, explained 67% of the variance in
balanced accuracy, vs. 54% for class balance and just 39% for size. The best
pair of measures was size-plus-$A_1$ (79%), which outperformed
size-plus-class-balance (74%). Subsets with the largest $A_0$ performed up to
16% better than those with the largest size (median improvement, 8%). We
propose maximizing $A$ as a way to improve deep learning performance in medical
imaging.
