  Multi-task learning (MTL) is an active field in deep learning in which we
train a model to jointly learn multiple tasks by exploiting relationships
between the tasks. It has been shown that MTL helps the model share the learned
features between tasks and enhance predictions compared to when learning each
task independently. We propose a new learning framework for 2-task MTL problem
that uses the predictions of one task as inputs to another network to predict
the other task. We define two new loss terms inspired by cycle-consistency loss
and contrastive learning, alignment loss and cross-task consistency loss. Both
losses are designed to enforce the model to align the predictions of multiple
tasks so that the model predicts consistently. We theoretically prove that both
losses help the model learn more efficiently and that cross-task consistency
loss is better in terms of alignment with the straight-forward predictions.
Experimental results also show that our proposed model achieves significant
performance on the benchmark Cityscapes and NYU dataset.
