  In this paper, we propose a novel method to incorporate partial evidence in
the inference of deep convolutional neural networks. Contrary to the existing,
top performing methods, which either iteratively modify the input of the
network or exploit external label taxonomy to take the partial evidence into
account, we add separate network modules ("Plugin Networks") to the
intermediate layers of a pre-trained convolutional network. The goal of these
modules is to incorporate additional signal, ie information about known labels,
into the inference procedure and adjust the predicted output accordingly. Since
the attached plugins have a simple structure, consisting of only fully
connected layers, we drastically reduced the computational cost of training and
inference. At the same time, the proposed architecture allows to propagate
information about known labels directly to the intermediate layers to improve
the final representation. Extensive evaluation of the proposed method confirms
that our Plugin Networks outperform the state-of-the-art in a variety of tasks,
including scene categorization, multi-label image annotation, and semantic
segmentation.
