  Establishing semantic correspondence across images when the objects in the
images have undergone complex deformations remains a challenging task in the
field of computer vision. In this paper, we propose a hierarchical method to
tackle this problem by first semantically targeting the foreground objects to
localize the search space and then looking deeply into multiple levels of the
feature representation to search for point-level correspondence. In contrast to
existing approaches, which typically penalize large discrepancies, our approach
allows for significant displacements, with the aim to accommodate large
deformations of the objects in scene. Localizing the search space by
semantically matching object-level correspondence, our method robustly handles
large deformations of objects. Representing the target region by concatenated
hypercolumn features which take into account the hierarchical levels of the
surrounding context, helps to clear the ambiguity to further improve the
accuracy. By conducting multiple experiments across scenes with non-rigid
objects, we validate the proposed approach, and show that it outperforms the
state of the art methods for semantic correspondence establishment.
