  This paper is concerned with the problem of how to better exploit 3D
geometric information for dense semantic image labeling. Existing methods often
treat the available 3D geometry information (e.g., 3D depth-map) simply as an
additional image channel besides the R-G-B color channels, and apply the same
technique for RGB image labeling. In this paper, we demonstrate that directly
performing 3D convolution in the framework of a residual connected 3D voxel
top-down modulation network can lead to superior results. Specifically, we
propose a 3D semantic labeling method to label outdoor street scenes whenever a
dense depth map is available. Experiments on the "Synthia" and "Cityscape"
datasets show our method outperforms the state-of-the-art methods, suggesting
such a simple 3D representation is effective in incorporating 3D geometric
information.
