  The community of program optimisation and analysis, code performance
evaluation, parallelisation and optimising compilation has published since many
decades hundreds of research and engineering articles in major conferences and
journals. These articles study efficient algorithms, strategies and techniques
to accelerate programs execution times, or optimise other performance metrics
(MIPS, code size, energy/power, MFLOPS, etc.). Many speedups are published, but
nobody is able to reproduce them exactly. The non-reproducibility of our
research results is a dark point of the art, and we cannot be qualified as {\it
computer scientists} if we do not provide rigorous experimental methodology.
This article provides a first effort towards a correct statistical protocol for
analysing and measuring speedups. As we will see, some common mistakes are done
by the community inside published articles, explaining part of the
non-reproducibility of the results. Our current article is not sufficient by
its own to deliver a complete experimental methodology, further efforts must be
done by the community to decide about a common protocol for our future
experiences. Anyway, our community should take care about the aspect of
reproducibility of the results in the future.
