  Moving Object Detection (MOD) is a critical vision task for successfully
achieving safe autonomous driving. Despite plausible results of deep learning
methods, most existing approaches are only frame-based and may fail to reach
reasonable performance when dealing with dynamic traffic participants. Recent
advances in sensor technologies, especially the Event camera, can naturally
complement the conventional camera approach to better model moving objects.
However, event-based works often adopt a pre-defined time window for event
representation, and simply integrate it to estimate image intensities from
events, neglecting much of the rich temporal information from the available
asynchronous events. Therefore, from a new perspective, we propose RENet, a
novel RGB-Event fusion Network, that jointly exploits the two complementary
modalities to achieve more robust MOD under challenging scenarios for
autonomous driving. Specifically, we first design a temporal multi-scale
aggregation module to fully leverage event frames from both the RGB exposure
time and larger intervals. Then we introduce a bi-directional fusion module to
attentively calibrate and fuse multi-modal features. To evaluate the
performance of our network, we carefully select and annotate a sub-MOD dataset
from the commonly used DSEC dataset. Extensive experiments demonstrate that our
proposed method performs significantly better than the state-of-the-art
RGB-Event fusion alternatives. The source code and dataset are publicly
available at: https://github.com/ZZY-Zhou/RENet.
