  We evaluated the effectiveness of an automated bird sound identification
system in a situation that emulates a realistic, typical application. We
trained classification algorithms on a crowd-sourced collection of bird audio
recording data and restricted our training methods to be completely free of
manual intervention. The approach is hence directly applicable to the analysis
of multiple species collections, with labelling provided by crowd-sourced
collection. We evaluated the performance of the bird sound recognition system
on a realistic number of candidate classes, corresponding to real conditions.
We investigated the use of two canonical classification methods, chosen due to
their widespread use and ease of interpretation, namely a k Nearest Neighbour
(kNN) classifier with histogram-based features and a Support Vector Machine
(SVM) with time-summarisation features. We further investigated the use of a
certainty measure, derived from the output probabilities of the classifiers, to
enhance the interpretability and reliability of the class decisions. Our
results demonstrate that both identification methods achieved similar
performance, but we argue that the use of the kNN classifier offers somewhat
more flexibility. Furthermore, we show that employing an outcome certainty
measure provides a valuable and consistent indicator of the reliability of
classification results. Our use of generic training data and our investigation
of probabilistic classification methodologies that can flexibly address the
variable number of candidate species/classes that are expected to be
encountered in the field, directly contribute to the development of a practical
bird sound identification system with potentially global application. Further,
we show that certainty measures associated with identification outcomes can
significantly contribute to the practical usability of the overall system.
