  Multiple-object tracking and behavior analysis have been the essential parts
of surveillance video analysis for public security and urban management. With
billions of surveillance video captured all over the world, multiple-object
tracking and behavior analysis by manual labor are cumbersome and cost
expensive. Due to the rapid development of deep learning algorithms in recent
years, automatic object tracking and behavior analysis put forward an urgent
demand on a large scale well-annotated surveillance video dataset that can
reflect the diverse, congested, and complicated scenarios in real applications.
This paper introduces an urban surveillance video dataset (USVD) which is by
far the largest and most comprehensive. The dataset consists of 16 scenes
captured in 7 typical outdoor scenarios: street, crossroads, hospital entrance,
school gate, park, pedestrian mall, and public square. Over 200k video frames
are annotated carefully, resulting in more than 3:7 million object bounding
boxes and about 7:1 thousand trajectories. We further use this dataset to
evaluate the performance of typical algorithms for multiple-object tracking and
anomaly behavior analysis and explore the robustness of these methods in urban
congested scenarios.
