  The art of finding software vulnerabilities has been covered extensively in
the literature and there is a huge body of work on this topic. In contrast, the
intentional insertion of exploitable, security-critical bugs has received
little (public) attention yet. Wanting more bugs seems to be counterproductive
at first sight, but the comprehensive evaluation of bug-finding techniques
suffers from a lack of ground truth and the scarcity of bugs.
  In this paper, we propose EvilCoder, a system to automatically find
potentially vulnerable source code locations and modify the source code to be
actually vulnerable. More specifically, we leverage automated program analysis
techniques to find sensitive sinks which match typical bug patterns (e.g., a
sensitive API function with a preceding sanity check), and try to find
data-flow connections to user-controlled sources. We then transform the source
code such that exploitation becomes possible, for example by removing or
modifying input sanitization or other types of security checks. Our tool is
designed to randomly pick vulnerable locations and possible modifications, such
that it can generate numerous different vulnerabilities on the same software
corpus. We evaluated our tool on several open-source projects such as for
example libpng and vsftpd, where we found between 22 and 158 unique connected
source-sink pairs per project. This translates to hundreds of potentially
vulnerable data-flow paths and hundreds of bugs we can insert. We hope to
support future bug-finding techniques by supplying freshly generated,
bug-ridden test corpora so that such techniques can (finally) be evaluated and
compared in a comprehensive and statistically meaningful way.
