  Given a reference object of an unknown type in an image, human observers can
effortlessly find the objects of the same category in another image and
precisely tell their visual boundaries. Such visual cognition capability of
humans seems absent from the current research spectrum of computer vision.
Existing segmentation networks, for example, rely on a humongous amount of
labeled data, which is laborious and costly to collect and annotate; besides,
the performance of segmentation networks tend to downgrade as the number of the
category increases. In this paper, we introduce a novel Reference semantic
segmentation Network (Ref-Net) to conduct visual boundary knowledge
translation. Ref-Net contains a Reference Segmentation Module (RSM) and a
Boundary Knowledge Translation Module (BKTM). Inspired by the human recognition
mechanism, RSM is devised only to segment the same category objects based on
the features of the reference objects. BKTM, on the other hand, introduces two
boundary discriminator branches to conduct inner and outer boundary
segmentation of the target objectin an adversarial manner, and translate the
annotated boundary knowledge of open-source datasets into the segmentation
network. Exhaustive experiments demonstrate that, with tens of finely-grained
annotated samples as guidance, Ref-Net achieves results on par with fully
supervised methods on six datasets.
