  Numerous open-source and commercial malware detectors are available. However,
their efficacy is threatened by new adversarial attacks, whereby malware
attempts to evade detection, e.g., by performing feature-space manipulation. In
this work, we propose an explainability-guided and model-agnostic testing
framework for robustness of malware detectors when confronted with adversarial
attacks. The framework introduces the concept of Accrued Malicious Magnitude
(AMM) to identify which malware features could be manipulated to maximize the
likelihood of evading detection. We then use this framework to test several
state-of-the-art malware detectors' abilities to detect manipulated malware. We
find that (i) commercial antivirus engines are vulnerable to AMM-guided test
cases; (ii) the ability of a manipulated malware generated using one detector
to evade detection by another detector (i.e., transferability) depends on the
overlap of features with large AMM values between the different detectors; and
(iii) AMM values effectively measure the fragility of features (i.e.,
capability of feature-space manipulation to flip the prediction results) and
explain the robustness of malware detectors facing evasion attacks. Our
findings shed light on the limitations of current malware detectors, as well as
how they can be improved.
