  Deep Recommender Systems (DRS) are increasingly dependent on a large number
of feature fields for more precise recommendations. Effective feature selection
methods are consequently becoming critical for further enhancing the accuracy
and optimizing storage efficiencies to align with the deployment demands. This
research area, particularly in the context of DRS, is nascent and faces three
core challenges. Firstly, variant experimental setups across research papers
often yield unfair comparisons, obscuring practical insights. Secondly, the
existing literature's lack of detailed analysis on selection attributes, based
on large-scale datasets and a thorough comparison among selection techniques
and DRS backbones, restricts the generalizability of findings and impedes
deployment on DRS. Lastly, research often focuses on comparing the peak
performance achievable by feature selection methods, an approach that is
typically computationally infeasible for identifying the optimal
hyperparameters and overlooks evaluating the robustness and stability of these
methods. To bridge these gaps, this paper presents ERASE, a comprehensive
bEnchmaRk for feAture SElection for DRS. ERASE comprises a thorough evaluation
of eleven feature selection methods, covering both traditional and deep
learning approaches, across four public datasets, private industrial datasets,
and a real-world commercial platform, achieving significant enhancement. Our
code is available online for ease of reproduction.
