  Referring expression comprehension (REC) and segmentation (RES) are two
highly-related tasks, which both aim at identifying the referent according to a
natural language expression. In this paper, we propose a novel Multi-task
Collaborative Network (MCN) to achieve a joint learning of REC and RES for the
first time. In MCN, RES can help REC to achieve better language-vision
alignment, while REC can help RES to better locate the referent. In addition,
we address a key challenge in this multi-task setup, i.e., the prediction
conflict, with two innovative designs namely, Consistency Energy Maximization
(CEM) and Adaptive Soft Non-Located Suppression (ASNLS). Specifically, CEM
enables REC and RES to focus on similar visual regions by maximizing the
consistency energy between two tasks. ASNLS supresses the response of unrelated
regions in RES based on the prediction of REC. To validate our model, we
conduct extensive experiments on three benchmark datasets of REC and RES, i.e.,
RefCOCO, RefCOCO+ and RefCOCOg. The experimental results report the significant
performance gains of MCN over all existing methods, i.e., up to +7.13% for REC
and +11.50% for RES over SOTA, which well confirm the validity of our model for
joint REC and RES learning.
