  We discuss a range of miscodes found in probabilistic genotyping (PG)
software and from other industries that have been reported in the literature
and have been used to inform PG admissibility hearings. Every instance of the
discovery of a miscode in PG software with which we have been associated has
occurred either because of testing, use, or repeat calculation of results
either by us or other users. In all cases found during testing or use something
has drawn attention to an anomalous result. Intelligent investigation has led
to the examination of a small section of the code and detection of the miscode.
Previously, three instances from other industries quoted by the Electronic
Frontier Foundation Amicus brief as part of a PG admissibility hearing
(atmospheric ozone, NIMIS, and VW) and two previous examples raised in relation
to PG admissibility (Kerberos and Therac-25) were presented as examples of
miscodes and how an extensive code review could have resolved these situations.
However, we discuss how these miscodes might not have been discovered through
code review alone. These miscodes could only have been detected through use of
the software or through testing. Once the symptoms of the miscode(s) have been
detected, a code review serves as a beneficial approach to try and diagnose to
the issue.
