  The ability to understand the surrounding scene is of paramount importance
for Autonomous Vehicles (AVs). This paper presents a system capable to work in
an online fashion, giving an immediate response to the arise of anomalies
surrounding the AV, exploiting only the videos captured by a dash-mounted
camera. Our architecture, called MOVAD, relies on two main modules: a
Short-Term Memory Module to extract information related to the ongoing action,
implemented by a Video Swin Transformer (VST), and a Long-Term Memory Module
injected inside the classifier that considers also remote past information and
action context thanks to the use of a Long-Short Term Memory (LSTM) network.
The strengths of MOVAD are not only linked to its excellent performance, but
also to its straightforward and modular architecture, trained in a end-to-end
fashion with only RGB frames with as less assumptions as possible, which makes
it easy to implement and play with. We evaluated the performance of our method
on Detection of Traffic Anomaly (DoTA) dataset, a challenging collection of
dash-mounted camera videos of accidents. After an extensive ablation study,
MOVAD is able to reach an AUC score of 82.17\%, surpassing the current
state-of-the-art by +2.87 AUC. Our code will be available on
https://github.com/IMPLabUniPr/movad/tree/movad_vad
