  Factor importance measures the impact of each feature on output prediction
accuracy. Many existing works focus on the model-based importance, but an
important feature in one learning algorithm may hold little significance in
another model. Hence, a factor importance measure ought to characterize the
feature's predictive potential without relying on a specific prediction
algorithm. Such algorithm-agnostic importance is termed as intrinsic importance
in Williamson et al. (2023), but their estimator again requires model fitting.
To bypass the modeling step, we present the equivalence between predictiveness
potential and total Sobol' indices from global sensitivity analysis, and
introduce a novel consistent estimator that can be directly estimated from
noisy data. Integrating with forward selection and backward elimination gives
rise to FIRST, Factor Importance Ranking and Selection using Total (Sobol')
indices. Extensive simulations are provided to demonstrate the effectiveness of
FIRST on regression and binary classification problems, and a clear advantage
over the state-of-the-art methods.
