  The security guarantee of AI-enabled software systems (particularly using
deep learning techniques as a functional core) is pivotal against the
adversarial attacks exploiting software vulnerabilities. However, little
attention has been paid to a systematic investigation of vulnerabilities in
such systems. A common situation learned from the open source software
community is that deep learning engineers frequently integrate off-the-shelf or
open-source learning frameworks into their ecosystems. In this work, we
specifically look into deep learning (DL) framework and perform the first
systematic study of vulnerabilities in DL systems through a comprehensive
analysis of identified vulnerabilities from Common Vulnerabilities and
Exposures (CVE) and open-source DL tools, including TensorFlow, Caffe, OpenCV,
Keras, and PyTorch. We propose a two-stream data analysis framework to explore
vulnerability patterns from various databases. We investigate the unique DL
frameworks and libraries development ecosystems that appear to be decentralized
and fragmented. By revisiting the Common Weakness Enumeration (CWE) List, which
provides the traditional software vulnerability related practices, we observed
that it is more challenging to detect and fix the vulnerabilities throughout
the DL systems lifecycle. Moreover, we conducted a large-scale empirical study
of 3,049 DL vulnerabilities to better understand the patterns of vulnerability
and the challenges in fixing them. We have released the full replication
package at https://github.com/codelzz/Vulnerabilities4DLSystem. We anticipate
that our study can advance the development of secure DL systems.
