  Machine learning algorithms are growing increasingly popular in particle
physics analyses, where they are used for their ability to solve difficult
classification and regression problems. While the tools are very powerful, they
may often be under- or mis-utilised. In the following, we investigate the use
of gradient boosting techniques as applicable to a generic particle physics
problem. We use as an example a Beyond the Standard Model smuon collider
analysis which applies to both current and future hadron colliders, and we
compare our results to a traditional cut-and-count approach. In particular, we
interrogate the use of metrics in imbalanced datasets which are characteristic
of high energy physics problems, offering an alternative to the widely used
area under the curve (auc) metric through a novel use of the F-score metric. We
present an in-depth comparison of feature selection and investigation using a
principal component analysis, Shapley values, and feature permutation methods
in a way which we hope will be widely applicable to future particle physics
analyses. Moreover, we show that a machine learning model can extend the 95%
confidence level exclusions obtained in a traditional cut-and-count analysis,
while potentially bypassing the need for complicated feature selections.
Finally, we discuss the possibility of constructing a general machine learning
model which is applicable to probe a two-dimensional mass plane.
