  In this paper, we propose a novel self-supervised representation learning by
taking advantage of a neighborhood-relational encoding (NRE) among the training
data. Conventional unsupervised learning methods only focused on training deep
networks to understand the primitive characteristics of the visual data, mainly
to be able to reconstruct the data from a latent space. They often neglected
the relation among the samples, which can serve as an important metric for
self-supervision. Different from the previous work, NRE aims at preserving the
local neighborhood structure on the data manifold. Therefore, it is less
sensitive to outliers. We integrate our NRE component with an encoder-decoder
structure for learning to represent samples considering their local
neighborhood information. Such discriminative and unsupervised representation
learning scheme is adaptable to different computer vision tasks due to its
independence from intense annotation requirements. We evaluate our proposed
method for different tasks, including classification, detection, and
segmentation based on the learned latent representations. In addition, we adopt
the auto-encoding capability of our proposed method for applications like
defense against adversarial example attacks and video anomaly detection.
Results confirm the performance of our method is better or at least comparable
with the state-of-the-art for each specific application, but with a generic and
self-supervised approach.
