  Agreeing suitability for purpose and procurement decisions depend on
assessment of real or simulated performances of sonar systems against user
requirements for particular scenarios. There may be multiple pertinent aspects
of performance (e.g. detection, track estimation, identification/classification
and cost) and multiple users (e.g. within picture compilation, threat
assessment, resource allocation and intercept control tasks), each with
different requirements. Further, the estimates of performances and the user
requirements are likely to be uncertain. In such circumstances, how can we
reliably assess and compare the effectiveness of candidate systems? This paper
presents a general yet simple mathematical framework that achieves all of this.
First, the general requirements of a satisfactory framework are outlined. Then,
starting from a definition of a measure of effectiveness (MOE) based on set
theory, the formulae for assessing performance in various applications are
obtained. These include combined MOEs, multiple and possibly conflicting user
requirements, multiple sources and types of performance data and different
descriptions of uncertainty. Issues raised by implementation of the scheme
within a simulator are discussed. Finally, it is shown how this approach to
performance assessment is used to treat some challenging examples from sonar
system assessment.
