  Traffic signs are critical for maintaining the safety and efficiency of our
roads. Therefore, we need to carefully assess the capabilities and limitations
of automated traffic sign detection systems. Existing traffic sign datasets are
limited in terms of type and severity of challenging conditions. Metadata
corresponding to these conditions are unavailable and it is not possible to
investigate the effect of a single factor because of simultaneous changes in
numerous conditions. To overcome the shortcomings in existing datasets, we
introduced the CURE-TSD-Real dataset, which is based on simulated challenging
conditions that correspond to adversaries that can occur in real-world
environments and systems. We test the performance of two benchmark algorithms
and show that severe conditions can result in an average performance
degradation of 29% in precision and 68% in recall. We investigate the effect of
challenging conditions through spectral analysis and show that challenging
conditions can lead to distinct magnitude spectrum characteristics. Moreover,
we show that mean magnitude spectrum of changes in video sequences under
challenging conditions can be an indicator of detection performance.
CURE-TSD-Real dataset is available online at
https://github.com/olivesgatech/CURE-TSD.
