  We propose a benchmark of state-of-the-art sound event detection systems
(SED). We designed synthetic evaluation sets to focus on specific sound event
detection challenges. We analyze the performance of the submissions to DCASE
2021 task 4 depending on time related modifications (time position of an event
and length of clips) and we study the impact of non-target sound events and
reverberation. We show that the localization in time of sound events is still a
problem for SED systems. We also show that reverberation and non-target sound
events are severely degrading the performance of the SED systems. In the latter
case, sound separation seems like a promising solution.
