  This paper addresses the problem of RGBD object recognition in real-world
applications, where large amounts of annotated training data are typically
unavailable. To overcome this problem, we propose a novel, weakly-supervised
learning architecture (DCNN-GPC) which combines parametric models (a pair of
Deep Convolutional Neural Networks (DCNN) for RGB and D modalities) with
non-parametric models (Gaussian Process Classification). Our system is
initially trained using a small amount of labeled data, and then automatically
prop- agates labels to large-scale unlabeled data. We first run 3D- based
objectness detection on RGBD videos to acquire many unlabeled object proposals,
and then employ DCNN-GPC to label them. As a result, our multi-modal DCNN can
be trained end-to-end using only a small amount of human annotation. Finally,
our 3D-based objectness detection and multi-modal DCNN are integrated into a
real-time detection and recognition pipeline. In our approach, bounding-box
annotations are not required and boundary-aware detection is achieved. We also
propose a novel way to pretrain a DCNN for the depth modality, by training on
virtual depth images projected from CAD models. We pretrain our multi-modal
DCNN on public 3D datasets, achieving performance comparable to
state-of-the-art methods on Washington RGBS Dataset. We then finetune the
network by further training on a small amount of annotated data from our novel
dataset of industrial objects (nuclear waste simulants). Our weakly supervised
approach has demonstrated to be highly effective in solving a novel RGBD object
recognition application which lacks of human annotations.
