  Due to the ever-increasing security breaches, practitioners are motivated to
produce more secure software. In the United States, the White House Office
released a memorandum on Executive Order (EO) 14028 that mandates organizations
provide self-attestation of the use of secure software development practices.
The OpenSSF Scorecard project allows practitioners to measure the use of
software security practices automatically. However, little research has been
done to determine whether the use of security practices improves package
security, particularly which security practices have the biggest impact on
security outcomes. The goal of this study is to assist practitioners and
researchers making informed decisions on which security practices to adopt
through the development of models between software security practice scores and
security vulnerability counts.
  To that end, we developed five supervised machine learning models for npm and
PyPI packages using the OpenSSF Scorecared security practices scores and
aggregate security scores as predictors and the number of externally-reported
vulnerabilities as a target variable. Our models found four security practices
(Maintained, Code Review, Branch Protection, and Security Policy) were the most
important practices influencing vulnerability count. However, we had low R^2
(ranging from 9% to 12%) when we tested the models to predict vulnerability
counts. Additionally, we observed that the number of reported vulnerabilities
increased rather than reduced as the aggregate security score of the packages
increased. Both findings indicate that additional factors may influence the
package vulnerability count. We suggest that vulnerability count and security
score data be refined such that these measures may be used to provide
actionable guidance on security practices.
