  3D sensing is a fundamental task for Autonomous Vehicles. Its deployment
often relies on aligned RGB cameras and LiDAR. Despite meticulous
synchronization and calibration, systematic misalignment persists in LiDAR
projected depthmap. This is due to the physical baseline distance between the
two sensors. The artifact is often reflected as background LiDAR incorrectly
projected onto the foreground, such as cars and pedestrians. The KITTI dataset
uses stereo cameras as a heuristic solution to remove artifacts. However most
AV datasets, including nuScenes, Waymo, and DDAD, lack stereo images, making
the KITTI solution inapplicable. We propose RePLAy, a parameter-free analytical
solution to remove the projective artifacts. We construct a binocular vision
system between a hypothesized virtual LiDAR camera and the RGB camera. We then
remove the projective artifacts by determining the epipolar occlusion with the
proposed analytical solution. We show unanimous improvement in the
State-of-The-Art (SoTA) monocular depth estimators and 3D object detectors with
the artifacts-free depthmaps.
