  The current state-of-the-art Android malware detection systems are based on
machine learning and deep learning models. Despite having superior performance,
these models are susceptible to adversarial attacks. Therefore in this paper,
we developed eight Android malware detection models based on machine learning
and deep neural network and investigated their robustness against adversarial
attacks. For this purpose, we created new variants of malware using
Reinforcement Learning, which will be misclassified as benign by the existing
Android malware detection models. We propose two novel attack strategies,
namely single policy attack and multiple policy attack using reinforcement
learning for white-box and grey-box scenario respectively. Putting ourselves in
the adversary's shoes, we designed adversarial attacks on the detection models
with the goal of maximizing fooling rate, while making minimum modifications to
the Android application and ensuring that the app's functionality and behavior
do not change. We achieved an average fooling rate of 44.21% and 53.20% across
all the eight detection models with a maximum of five modifications using a
single policy attack and multiple policy attack, respectively. The highest
fooling rate of 86.09% with five changes was attained against the decision
tree-based model using the multiple policy approach. Finally, we propose an
adversarial defense strategy that reduces the average fooling rate by threefold
to 15.22% against a single policy attack, thereby increasing the robustness of
the detection models i.e. the proposed model can effectively detect variants
(metamorphic) of malware. The experimental analysis shows that our proposed
Android malware detection system using reinforcement learning is more robust
against adversarial attacks.
