  We address the challenging problem of semi-supervised learning in the context
of multiple visual interpretations of the world by finding consensus in a graph
of neural networks. Each graph node is a scene interpretation layer, while each
edge is a deep net that transforms one layer at one node into another from a
different node. During the supervised phase edge networks are trained
independently. During the next unsupervised stage edge nets are trained on the
pseudo-ground truth provided by consensus among multiple paths that reach the
nets' start and end nodes. These paths act as ensemble teachers for any given
edge and strong consensus is used for high-confidence supervisory signal. The
unsupervised learning process is repeated over several generations, in which
each edge becomes a "student" and also part of different ensemble "teachers"
for training other students. By optimizing such consensus between different
paths, the graph reaches consistency and robustness over multiple
interpretations and generations, in the face of unknown labels. We give
theoretical justifications of the proposed idea and validate it on a large
dataset. We show how prediction of different representations such as depth,
semantic segmentation, surface normals and pose from RGB input could be
effectively learned through self-supervised consensus in our graph. We also
compare to state-of-the-art methods for multi-task and semi-supervised learning
and show superior performance.
