  As researchers and practitioners of applied machine learning, we are given a
set of requirements on the problem to be solved, the plausibly obtainable data,
and the computational resources available. We aim to find (within those bounds)
reliably useful combinations of problem, data, and algorithm. An emphasis on
algorithmic or technical novelty in ML conference publications leads to
exploration of one dimension of this space. Data collection and ML deployment
at scale in industry settings offers an environment for exploring the others.
Our conferences and reviewing criteria can better support empirical ML by
soliciting and incentivizing experimentation and synthesis independent of
algorithmic innovation.
