  A single rolling-shutter (RS) image may be viewed as a row-wise combination
of a sequence of global-shutter (GS) images captured by a (virtual) moving GS
camera within the exposure duration. Although RS cameras are widely used, the
RS effect causes obvious image distortion especially in the presence of fast
camera motion, hindering downstream computer vision tasks. In this paper, we
propose to invert the RS image capture mechanism, i.e., recovering a continuous
high framerate GS video from two time-consecutive RS frames. We call this task
the RS temporal super-resolution (RSSR) problem. The RSSR is a very challenging
task, and to our knowledge, no practical solution exists to date. This paper
presents a novel deep-learning based solution. By leveraging the multi-view
geometry relationship of the RS imaging process, our learning-based framework
successfully achieves high framerate GS generation. Specifically, three novel
contributions can be identified: (i) novel formulations for bidirectional RS
undistortion flows under constant velocity as well as constant acceleration
motion model. (ii) a simple linear scaling operation, which bridges the RS
undistortion flow and regular optical flow. (iii) a new mutual conversion
scheme between varying RS undistortion flows that correspond to different
scanlines. Our method also exploits the underlying spatial-temporal geometric
relationships within a deep learning framework, where no additional supervision
is required beyond the necessary middle-scanline GS image. Building upon these
contributions, we represent the very first rolling-shutter temporal
super-resolution deep-network that is able to recover high framerate GS videos
from just two RS frames. Extensive experimental results on both synthetic and
real data show that our proposed method can produce high-quality GS image
sequences with rich details, outperforming the state-of-the-art methods.
