  Researchers at the Ames Laboratory-USDOE and the Federal Bureau of
Investigation (FBI) conducted a study to assess the performance of forensic
examiners in firearm investigations. The study involved three different types
of firearms and 173 volunteers who compared both bullets and cartridge cases.
The total number of comparisons reported is 20,130, allocated to assess
accuracy (8,640), repeatability (5,700), and reproducibility (5,790) of the
evaluations made by participating examiners. The overall false positive error
rate was estimated as 0.656% and 0.933% for bullets and cartridge cases,
respectively, while the rate of false negatives was estimated as 2.87% and
1.87% for bullets and cartridge cases, respectively. Because chi-square tests
of independence strongly suggest that error probabilities are not the same for
each examiner, these are maximum likelihood estimates based on the
beta-binomial probability model and do not depend on an assumption of equal
examiner-specific error rates. Corresponding 95% confidence intervals are
(0.305%,1.42%) and (0.548%,1.57%) for false positives for bullets and cartridge
cases, respectively, and (1.89%,4.26%) and (1.16%,2.99%) for false negatives
for bullets and cartridge cases, respectively. These results are based on data
representing all controlled conditions considered, including different firearm
manufacturers, sequence of manufacture, and firing separation between unknown
and known comparison specimens. The results are consistent with those of prior
studies, despite its more robust design and challenging specimens.
