  Traffic anomaly detection has played a crucial role in Intelligent
Transportation System (ITS). The main challenges of this task lie in the highly
diversified anomaly scenes and variational lighting conditions. Although much
work has managed to identify the anomaly in homogenous weather and scene, few
resolved to cope with complex ones. In this paper, we proposed a dual-modality
modularized methodology for the robust detection of abnormal vehicles. We
introduced an integrated anomaly detection framework comprising the following
modules: background modeling, vehicle tracking with detection, mask
construction, Region of Interest (ROI) backtracking, and dual-modality tracing.
Concretely, we employed background modeling to filter the motion information
and left the static information for later vehicle detection. For the vehicle
detection and tracking module, we adopted YOLOv5 and multi-scale tracking to
localize the anomalies. Besides, we utilized the frame difference and tracking
results to identify the road and obtain the mask. In addition, we introduced
multiple similarity estimation metrics to refine the anomaly period via
backtracking. Finally, we proposed a dual-modality bilateral tracing module to
refine the time further. The experiments conducted on the Track 4 testset of
the NVIDIA 2021 AI City Challenge yielded a result of 0.9302 F1-Score and
3.4039 root mean square error (RMSE), indicating the effectiveness of our
framework.
