  This paper argues for the widest possible use of bootstrap confidence
intervals for comparing NLP system performances instead of the state-of-the-art
status (SOTA) and statistical significance testing. Their main benefits are to
draw attention to the difference in performance between two systems and to help
assessing the degree of superiority of one system over another. Two cases
studies, one comparing several systems and the other based on a K-fold
cross-validation procedure, illustrate these benefits. A python module for
obtaining these confidence intervals as well as a second function implementing
the Fisher-Pitman test for paired samples are freely available on PyPi.
