  We warn against a common but incomplete understanding of empirical research
in machine learning that leads to non-replicable results, makes findings
unreliable, and threatens to undermine progress in the field. To overcome this
alarming situation, we call for more awareness of the plurality of ways of
gaining knowledge experimentally but also of some epistemic limitations. In
particular, we argue most current empirical machine learning research is
fashioned as confirmatory research while it should rather be considered
exploratory.
