  This paper is a technical report about our submission for the ECCV 2018 3DRMS
Workshop Challenge on Semantic 3D Reconstruction \cite{Tylecek2018rms}. In this
paper, we address 3D semantic reconstruction for autonomous navigation using
co-learning of depth map and semantic segmentation. The core of our pipeline is
a deep multi-task neural network which tightly refines depth and also produces
accurate semantic segmentation maps. Its inputs are an image and a raw depth
map produced from a pair of images by standard stereo vision. The resulting
semantic 3D point clouds are then merged in order to create a consistent 3D
mesh, in turn used to produce dense semantic 3D reconstruction maps. The
performances of each step of the proposed method are evaluated on the dataset
and multiple tasks of the 3DRMS Challenge, and repeatedly surpass
state-of-the-art approaches.
