  Scene Dynamic Recovery (SDR) by inverting distorted Rolling Shutter (RS)
images to an undistorted high frame-rate Global Shutter (GS) video is a
severely ill-posed problem due to the missing temporal dynamic information in
both RS intra-frame scanlines and inter-frame exposures, particularly when
prior knowledge about camera/object motions is unavailable. Commonly used
artificial assumptions on scenes/motions and data-specific characteristics are
prone to producing sub-optimal solutions in real-world scenarios. To address
this challenge, we propose an event-based SDR network within a self-supervised
learning paradigm, i.e., SelfUnroll. We leverage the extremely high temporal
resolution of event cameras to provide accurate inter/intra-frame dynamic
information. Specifically, an Event-based Inter/intra-frame Compensator (E-IC)
is proposed to predict the per-pixel dynamic between arbitrary time intervals,
including the temporal transition and spatial translation. Exploring
connections in terms of RS-RS, RS-GS, and GS-RS, we explicitly formulate mutual
constraints with the proposed E-IC, resulting in supervisions without
ground-truth GS images. Extensive evaluations over synthetic and real datasets
demonstrate that the proposed method achieves state-of-the-art and shows
remarkable performance for event-based RS2GS inversion in real-world scenarios.
The dataset and code are available at https://w3un.github.io/selfunroll/.
