  Since 2009, the deep learning revolution, which was triggered by the
introduction of ImageNet, has stimulated the synergy between Machine Learning
(ML)/Deep Learning (DL) and Software Engineering (SE). Meanwhile, critical
reviews have emerged that suggest that ML/DL should be used cautiously. To
improve the quality (especially the applicability and generalizability) of
ML/DL-related SE studies, and to stimulate and enhance future collaborations
between SE/AI researchers and industry practitioners, we conducted a 10-year
Systematic Literature Review (SLR) on 906 ML/DL-related SE papers published
between 2009 and 2018. Our trend analysis demonstrated the mutual impacts that
ML/DL and SE have had on each other. At the same time, however, we also
observed a paucity of replicable and reproducible ML/DL-related SE studies and
identified five factors that influence their replicability and reproducibility.
To improve the applicability and generalizability of research results, we
analyzed what ingredients in a study would facilitate an understanding of why a
ML/DL technique was selected for a specific SE problem. In addition, we
identified the unique trends of impacts of DL models on SE tasks, as well as
five unique challenges that needed to be met in order to better leverage DL to
improve the productivity of SE tasks. Finally, we outlined a road-map that we
believe can facilitate the transfer of ML/DL-based SE research results into
real-world industry practices.
