  We point out important problems with the common practice of using the best
single model performance for comparing deep learning architectures, and we
propose a method that corrects these flaws. Each time a model is trained, one
gets a different result due to random factors in the training process, which
include random parameter initialization and random data shuffling. Reporting
the best single model performance does not appropriately address this
stochasticity. We propose a normalized expected best-out-of-$n$ performance
($\text{Boo}_n$) as a way to correct these problems.
