  For regression tasks one often leverages large datasets for training
predictive machine learning models. However, using large datasets may not be
feasible due to computational limitations or high data labelling costs.
Therefore, suitably selecting small training sets from large pools of
unlabelled data points is essential to maximize model performance while
maintaining efficiency. In this work, we study Farthest Point Sampling (FPS), a
data selection approach that aims to minimize the fill distance of the selected
set. We derive an upper bound for the maximum expected prediction error,
conditional to the location of the unlabelled data points, that linearly
depends on the training set fill distance. For empirical validation, we perform
experiments using two regression models on three datasets. We empirically show
that selecting a training set by aiming to minimize the fill distance, thereby
minimizing our derived bound, significantly reduces the maximum prediction
error of various regression models, outperforming alternative sampling
approaches by a large margin. Furthermore, we show that selecting training sets
with the FPS can also increase model stability for the specific case of
Gaussian kernel regression approaches.
