  The ranking of sound event detection (SED) systems may be biased by
assumptions inherent to evaluation criteria and to the choice of an operating
point. This paper compares conventional event-based and segment-based criteria
against the Polyphonic Sound Detection Score (PSDS)'s intersection-based
criterion, over a selection of systems from DCASE 2020 Challenge Task 4. It
shows that, by relying on collars , the conventional event-based criterion
introduces different strictness levels depending on the length of the sound
events, and that the segment-based criterion may lack precision and be
application dependent. Alternatively, PSDS's intersection-based criterion
overcomes the dependency of the evaluation on sound event duration and provides
robustness to labelling subjectivity, by allowing valid detections of
interrupted events. Furthermore, PSDS enhances the comparison of SED systems by
measuring sound event modelling performance independently from the systems'
operating points.
