  Depth maps produced by consumer-grade sensors suffer from inaccurate
measurements and missing data from either system or scene-specific sources.
Data-driven denoising algorithms can mitigate such problems. However, they
require vast amounts of ground truth depth data. Recent research has tackled
this limitation using self-supervised learning techniques, but it requires
multiple RGB-D sensors. Moreover, most existing approaches focus on denoising
single isolated depth maps or specific subjects of interest, highlighting a
need for methods to effectively denoise depth maps in real-time dynamic
environments. This paper extends state-of-the-art approaches for
depth-denoising commodity depth devices, proposing SelfReDepth, a
self-supervised deep learning technique for depth restoration, via denoising
and hole-filling by inpainting full-depth maps captured with RGB-D sensors. The
algorithm targets depth data in video streams, utilizing multiple sequential
depth frames coupled with color data to achieve high-quality depth videos with
temporal coherence. Finally, SelfReDepth is designed to be compatible with
various RGB-D sensors and usable in real-time scenarios as a pre-processing
step before applying other depth-dependent algorithms. Our results demonstrate
our approach's real-time performance on real-world datasets. They show that it
outperforms state-of-the-art denoising and restoration performance at over
30fps on Commercial Depth Cameras, with potential benefits for augmented and
mixed-reality applications.
