  The popularity of Android means it is a common target for malware. Over the
years, various studies have found that machine learning models can effectively
discriminate malware from benign applications. However, as the operating system
evolves, so does malware, bringing into question the findings of these previous
studies, many of which report very high accuracies using small, outdated, and
often imbalanced datasets. In this paper, we reimplement 18 representative past
works and reevaluate them using a balanced, relevant, and up-to-date dataset
comprising 124,000 applications. We also carry out new experiments designed to
fill holes in existing knowledge, and use our findings to identify the most
effective features and models to use for Android malware detection within a
contemporary environment. We show that high detection accuracies (up to 96.8%)
can be achieved using features extracted through static analysis alone,
yielding a modest benefit (1%) from using far more expensive dynamic analysis.
API calls and opcodes are the most productive static and TCP network traffic
provide the most predictive dynamic features. Random forests are generally the
most effective model, outperforming more complex deep learning approaches.
Whilst directly combining static and dynamic features is generally ineffective,
ensembling models separately leads to performances comparable to the best
models but using less brittle features.
