  Traditional approaches to safety event analysis in autonomous systems have
relied on complex machine learning models and extensive datasets for high
accuracy and reliability. However, the advent of Multimodal Large Language
Models (MLLMs) offers a novel approach by integrating textual, visual, and
audio modalities, thereby providing automated analyses of driving videos. Our
framework leverages the reasoning power of MLLMs, directing their output
through context-specific prompts to ensure accurate, reliable, and actionable
insights for hazard detection. By incorporating models like Gemini-Pro-Vision
1.5 and Llava, our methodology aims to automate the safety critical events and
mitigate common issues such as hallucinations in MLLM outputs. Preliminary
results demonstrate the framework's potential in zero-shot learning and
accurate scenario analysis, though further validation on larger datasets is
necessary. Furthermore, more investigations are required to explore the
performance enhancements of the proposed framework through few-shot learning
and fine-tuned models. This research underscores the significance of MLLMs in
advancing the analysis of the naturalistic driving videos by improving
safety-critical event detecting and understanding the interaction with complex
environments.
