  This paper performs the first study to understand the prevalence, challenges,
and effectiveness of using Static Application Security Testing (SAST) tools on
Open-Source Embedded Software (EMBOSS) repositories. We collect a corpus of 258
of the most popular EMBOSS projects, representing 13 distinct categories such
as real-time operating systems, network stacks, and applications. To understand
the current use of SAST tools on EMBOSS, we measured this corpus and surveyed
developers. To understand the challenges and effectiveness of using SAST tools
on EMBOSS projects, we applied these tools to the projects in our corpus. We
report that almost none of these projects (just 3%) use SAST tools beyond those
baked into the compiler, and developers give rationales such as ineffectiveness
and false positives. In applying SAST tools ourselves, we show that minimal
engineering effort and project expertise are needed to apply many tools to a
given EMBOSS project. GitHub's CodeQL was the most effective SAST tool -- using
its built-in security checks we found a total of 540 defects (with a false
positive rate of 23%) across the 258 projects, with 399 (74%) likely security
vulnerabilities, including in projects maintained by Microsoft, Amazon, and the
Apache Foundation. EMBOSS engineers have confirmed 273 (51%) of these defects,
mainly by accepting our pull requests. Two CVEs were issued. In summary, we
urge EMBOSS engineers to adopt the current generation of SAST tools, which
offer low false positive rates and are effective at finding security-relevant
defects.
