  Existing real-time RGB-D reconstruction approaches, like Kinect Fusion, lack
real-time photo-realistic visualization. This is due to noisy, oversmoothed or
incomplete geometry and blurry textures which are fused from imperfect depth
maps and camera poses. Recent neural rendering methods can overcome many of
such artifacts but are mostly optimized for offline usage, hindering the
integration into a live reconstruction pipeline.
  In this paper, we present LiveNVS, a system that allows for neural novel view
synthesis on a live RGB-D input stream with very low latency and real-time
rendering. Based on the RGB-D input stream, novel views are rendered by
projecting neural features into the target view via a densely fused depth map
and aggregating the features in image-space to a target feature map. A
generalizable neural network then translates the target feature map into a
high-quality RGB image. LiveNVS achieves state-of-the-art neural rendering
quality of unknown scenes during capturing, allowing users to virtually explore
the scene and assess reconstruction quality in real-time.
