  Model selection requires repeatedly evaluating models on a given dataset and
measuring their relative performances. In modern applications of machine
learning, the models being considered are increasingly more expensive to
evaluate and the datasets of interest are increasing in size. As a result, the
process of model selection is time-consuming and computationally inefficient.
In this work, we develop a model-specific data subsampling strategy that
improves over random sampling whenever training points have varying influence.
Specifically, we leverage influence functions to guide our selection strategy,
proving theoretically, and demonstrating empirically that our approach quickly
selects high-quality models.
