  Estimating the importance of variables is an essential task in modern machine
learning. This help to evaluate the goodness of a feature in a given model.
Several techniques for estimating the importance of variables have been
developed during the last decade. In this paper, we proposed a computational
and theoretical exploration of the emerging methods of variable importance
estimation, namely: Least Absolute Shrinkage and Selection Operator (LASSO),
Support Vector Machine (SVM), the Predictive Error Function (PERF), Random
Forest (RF), and Extreme Gradient Boosting (XGBOOST) that were tested on
different kinds of real-life and simulated data. All these methods can handle
both regression and classification tasks seamlessly but all fail when it comes
to dealing with data containing missing values. The implementation has shown
that PERF has the best performance in the case of highly correlated data
closely followed by RF. PERF and XGBOOST are "data-hungry" methods, they had
the worst performance on small data sizes but they are the fastest when it
comes to the execution time. SVM is the most appropriate when many redundant
features are in the dataset. A surplus with the PERF is its natural cut-off at
zero helping to separate positive and negative scores with all positive scores
indicating essential and significant features while the negatives score
indicates useless features. RF and LASSO are very versatile in a way that they
can be used in almost all situations despite they are not giving the best
results.
