  Recent trends in the software development practices (Agile, DevOps, CI) have
shortened the development life-cycle causing the need for efficient
security-by-design approaches. In this context, software architectures are
analyzed for potential vulnerabilities and design flaws. Yet, design flaws are
often documented with natural language and require a manual analysis, which is
inefficient. Besides low-level vulnerability databases (e.g., CWE, CAPEC) there
is little systematized knowledge on security design flaws. The purpose of this
work is to provide a catalog of security design flaws and to empirically
evaluate the inspection guidelines for detecting security design flaws. To this
aim, we present a catalog of 19 security design flaws and conduct empirical
studies with master and doctoral students. This paper contributes with: (i) a
catalog of security design flaws, (ii) an empirical evaluation of the
inspection guidelines with master students, and (iii) a replicated evaluation
with doctoral students. We also account for the shortcomings of the inspection
guidelines and make suggestions for their improvement with respect to the
generalization of guidelines, catalog re-organization, and format of
documentation. We record similar precision, recall, and productivity in both
empirical studies and discuss the potential for automating the security design
flaw detection.
