  Vision sensors are widely applied in vehicles, robots, and roadside
infrastructure. However, due to limitations in hardware cost and system size,
camera Field-of-View (FoV) is often restricted and may not provide sufficient
coverage. Nevertheless, from a spatiotemporal perspective, it is possible to
obtain information beyond the camera's physical FoV from past video streams. In
this paper, we propose the concept of online video inpainting for autonomous
vehicles to expand the field of view, thereby enhancing scene visibility,
perception, and system safety. To achieve this, we introduce the FlowLens
architecture, which explicitly employs optical flow and implicitly incorporates
a novel clip-recurrent transformer for feature propagation. FlowLens offers two
key features: 1) FlowLens includes a newly designed Clip-Recurrent Hub with
3D-Decoupled Cross Attention (DDCA) to progressively process global information
accumulated over time. 2) It integrates a multi-branch Mix Fusion Feed Forward
Network (MixF3N) to enhance the precise spatial flow of local features. To
facilitate training and evaluation, we derive the KITTI360 dataset with various
FoV mask, which covers both outer- and inner FoV expansion scenarios. We also
conduct both quantitative assessments and qualitative comparisons of beyond-FoV
semantics and beyond-FoV object detection across different models. We
illustrate that employing FlowLens to reconstruct unseen scenes even enhances
perception within the field of view by providing reliable semantic context.
Extensive experiments and user studies involving offline and online video
inpainting, as well as beyond-FoV perception tasks, demonstrate that FlowLens
achieves state-of-the-art performance. The source code and dataset are made
publicly available at https://github.com/MasterHow/FlowLens.
