  Event cameras are novel sensors that perceive the per-pixel intensity changes
and output asynchronous event streams with high dynamic range and less motion
blur. It has been shown that events alone can be used for end-task learning,
e.g., semantic segmentation, based on encoder-decoder-like networks. However,
as events are sparse and mostly reflect edge information, it is difficult to
recover original details merely relying on the decoder. Moreover, most methods
resort to pixel-wise loss alone for supervision, which might be insufficient to
fully exploit the visual details from sparse events, thus leading to less
optimal performance. In this paper, we propose a simple yet flexible two-stream
framework named Dual Transfer Learning (DTL) to effectively enhance the
performance on the end-tasks without adding extra inference cost. The proposed
approach consists of three parts: event to end-task learning (EEL) branch,
event to image translation (EIT) branch, and transfer learning (TL) module that
simultaneously explores the feature-level affinity information and pixel-level
knowledge from the EIT branch to improve the EEL branch. This simple yet novel
method leads to strong representation learning from events and is evidenced by
the significant performance boost on the end-tasks such as semantic
segmentation and depth estimation.
