  Event cameras do not produce images, but rather a continuous flow of events,
which encode changes of illumination for each pixel independently and
asynchronously. While they output temporally rich information, they lack any
depth information which could facilitate their use with other sensors. LiDARs
can provide this depth information, but are by nature very sparse, which makes
the depth-to-event association more complex. Furthermore, as events represent
changes of illumination, they might also represent changes of depth;
associating them with a single depth is therefore inadequate. In this work, we
propose to address these issues by fusing information from an event camera and
a LiDAR using a learning-based approach to estimate accurate dense depth maps.
To solve the "potential change of depth" problem, we propose here to estimate
two depth maps at each step: one "before" the events happen, and one "after"
the events happen. We further propose to use this pair of depths to compute a
depth difference for each event, to give them more context. We train and
evaluate our network, ALED, on both synthetic and real driving sequences, and
show that it is able to predict dense depths with an error reduction of up to
61% compared to the current state of the art. We also demonstrate the quality
of our 2-depths-to-event association, and the usefulness of the depth
difference information. Finally, we release SLED, a novel synthetic dataset
comprising events, LiDAR point clouds, RGB images, and dense depth maps.
