  In this paper, we present a toolchain for a comprehensive audio/video
analysis by leveraging deep learning based multimodal approach. To this end,
different specific tasks of Speech to Text (S2T), Acoustic Scene Classification
(ASC), Acoustic Event Detection (AED), Visual Object Detection (VOD), Image
Captioning (IC), and Video Captioning (VC) are conducted and integrated into
the toolchain. By combining individual tasks and analyzing both audio \& visual
data extracted from input video, the toolchain offers various audio/video-based
applications: Two general applications of audio/video clustering, comprehensive
audio/video summary and a specific application of riot or violent context
detection. Furthermore, the toolchain presents a flexible and adaptable
architecture that is effective to integrate new models for further
audio/video-based applications.
