  Android malware still represents the most significant threat to mobile
systems. While Machine Learning systems are increasingly used to identify these
threats, past studies have revealed that attackers can bypass these detection
mechanisms by making subtle changes to Android applications, such as adding
specific API calls. These modifications are often referred to as No OPerations
(NOP), which ideally should not alter the semantics of the program. However,
many NOPs can be spotted and eliminated by refining the app analysis process.
This paper proposes a visibility metric that assesses the difficulty in
spotting NOPs and similar non-operational codes. We tested our metric on a
state-of-the-art, opcode-based deep learning system for Android malware
detection. We implemented attacks on the feature and problem spaces and
calculated their visibility according to our metric. The attained results show
an intriguing trade-off between evasion efficacy and detectability: our metric
can be valuable to ensure the real effectiveness of an adversarial attack, also
serving as a useful aid to develop better defenses.
