  RGB-D semantic segmentation can be advanced with convolutional neural
networks due to the availability of Depth data. Although objects cannot be
easily discriminated by just the 2D appearance, with the local pixel difference
and geometric patterns in Depth, they can be well separated in some cases.
Considering the fixed grid kernel structure, CNNs are limited to lack the
ability to capture detailed, fine-grained information and thus cannot achieve
accurate pixel-level semantic segmentation. To solve this problem, we propose a
Pixel Difference Convolutional Network (PDCNet) to capture detailed intrinsic
patterns by aggregating both intensity and gradient information in the local
range for Depth data and global range for RGB data, respectively. Precisely,
PDCNet consists of a Depth branch and an RGB branch. For the Depth branch, we
propose a Pixel Difference Convolution (PDC) to consider local and detailed
geometric information in Depth data via aggregating both intensity and gradient
information. For the RGB branch, we contribute a lightweight Cascade Large
Kernel (CLK) to extend PDC, namely CPDC, to enjoy global contexts for RGB data
and further boost performance. Consequently, both modal data's local and global
pixel differences are seamlessly incorporated into PDCNet during the
information propagation process. Experiments on two challenging benchmark
datasets, i.e., NYUDv2 and SUN RGB-D reveal that our PDCNet achieves
state-of-the-art performance for the semantic segmentation task.
