  Individual abnormal behaviors vary depending on crowd sizes, contexts, and
scenes. Challenges such as partial occlusions, blurring, large-number abnormal
behavior, and camera viewing occur in large-scale crowds when detecting,
tracking, and recognizing individuals with abnormal behaviors. In this paper,
our contribution is twofold. First, we introduce an annotated and labeled
large-scale crowd abnormal behaviors Hajj dataset (HAJJv2). Second, we propose
two methods of hybrid Convolutional Neural Networks (CNNs) and Random Forests
(RFs) to detect and recognize Spatio-temporal abnormal behaviors in small and
large-scales crowd videos. In small-scale crowd videos, a ResNet-50 pre-trained
CNN model is fine-tuned to verify whether every frame is normal or abnormal in
the spatial domain. If anomalous behaviors are observed, a motion-based
individuals detection method based on the magnitudes and orientations of
Horn-Schunck optical flow is used to locate and track individuals with abnormal
behaviors. A Kalman filter is employed in large-scale crowd videos to predict
and track the detected individuals in the subsequent frames. Then, means,
variances, and standard deviations statistical features are computed and fed to
the RF to classify individuals with abnormal behaviors in the temporal domain.
In large-scale crowds, we fine-tune the ResNet-50 model using YOLOv2 object
detection technique to detect individuals with abnormal behaviors in the
spatial domain.
