  Motivated by the need for photo-realistic simulation in autonomous driving,
in this paper we present a video inpainting algorithm \emph{AutoRemover},
designed specifically for generating street-view videos without any moving
objects. In our setup we have two challenges: the first is the shadow, shadows
are usually unlabeled but tightly coupled with the moving objects. The second
is the large ego-motion in the videos. To deal with shadows, we build up an
autonomous driving shadow dataset and design a deep neural network to detect
shadows automatically. To deal with large ego-motion, we take advantage of the
multi-source data, in particular the 3D data, in autonomous driving. More
specifically, the geometric relationship between frames is incorporated into an
inpainting deep neural network to produce high-quality structurally consistent
video output. Experiments show that our method outperforms other
state-of-the-art (SOTA) object removal algorithms, reducing the RMSE by over
$19\%$.
