  Visual perception entails solving a wide set of tasks, e.g., object
detection, depth estimation, etc. The predictions made for multiple tasks from
the same image are not independent, and therefore, are expected to be
consistent. We propose a broadly applicable and fully computational method for
augmenting learning with Cross-Task Consistency. The proposed formulation is
based on inference-path invariance over a graph of arbitrary tasks. We observe
that learning with cross-task consistency leads to more accurate predictions
and better generalization to out-of-distribution inputs. This framework also
leads to an informative unsupervised quantity, called Consistency Energy, based
on measuring the intrinsic consistency of the system. Consistency Energy
correlates well with the supervised error (r=0.67), thus it can be employed as
an unsupervised confidence metric as well as for detection of
out-of-distribution inputs (ROC-AUC=0.95). The evaluations are performed on
multiple datasets, including Taskonomy, Replica, CocoDoom, and ApolloScape, and
they benchmark cross-task consistency versus various baselines including
conventional multi-task learning, cycle consistency, and analytical
consistency.
