  Dataset scaling, also known as normalization, is an essential preprocessing
step in a machine learning pipeline. It is aimed at adjusting attributes scales
in a way that they all vary within the same range. This transformation is known
to improve the performance of classification models, but there are several
scaling techniques to choose from, and this choice is not generally done
carefully. In this paper, we execute a broad experiment comparing the impact of
5 scaling techniques on the performances of 20 classification algorithms among
monolithic and ensemble models, applying them to 82 publicly available datasets
with varying imbalance ratios. Results show that the choice of scaling
technique matters for classification performance, and the performance
difference between the best and the worst scaling technique is relevant and
statistically significant in most cases. They also indicate that choosing an
inadequate technique can be more detrimental to classification performance than
not scaling the data at all. We also show how the performance variation of an
ensemble model, considering different scaling techniques, tends to be dictated
by that of its base model. Finally, we discuss the relationship between a
model's sensitivity to the choice of scaling technique and its performance and
provide insights into its applicability on different model deployment
scenarios. Full results and source code for the experiments in this paper are
available in a GitHub
repository.\footnote{https://github.com/amorimlb/scaling\_matters}
