  (U.S) Rule-based policies to mitigate software risk suggest to use the CVSS
score to measure the individual vulnerability risk and act accordingly: an HIGH
CVSS score according to the NVD (National (U.S.) Vulnerability Database) is
therefore translated into a "Yes". A key issue is whether such rule is
economically sensible, in particular if reported vulnerabilities have been
actually exploited in the wild, and whether the risk score do actually match
the risk of actual exploitation.
  We compare the NVD dataset with two additional datasets, the EDB for the
white market of vulnerabilities (such as those present in Metasploit), and the
EKITS for the exploits traded in the black market. We benchmark them against
Symantec's threat explorer dataset (SYM) of actual exploit in the wild. We
analyze the whole spectrum of CVSS submetrics and use these characteristics to
perform a case-controlled analysis of CVSS scores (similar to those used to
link lung cancer and smoking) to test its reliability as a risk factor for
actual exploitation.
  We conclude that (a) fixing just because a high CVSS score in NVD only yields
negligible risk reduction, (b) the additional existence of proof of concepts
exploits (e.g. in EDB) may yield some additional but not large risk reduction,
(c) fixing in response to presence in black markets yields the equivalent risk
reduction of wearing safety belt in cars (you might also die but still..). On
the negative side, our study shows that as industry we miss a metric with high
specificity (ruling out vulns for which we shouldn't worry).
  In order to address the feedback from BlackHat 2013's audience, the final
revision (V3) provides additional data in Appendix A detailing how the control
variables in the study affect the results.
