  This paper introduces a novel deep learning based method, named bridge neural
network (BNN) to dig the potential relationship between two given data sources
task by task. The proposed approach employs two convolutional neural networks
that project the two data sources into a feature space to learn the desired
common representation required by the specific task. The training objective
with artificial negative samples is introduced with the ability of mini-batch
training and it's asymptotically equivalent to maximizing the total correlation
of the two data sources, which is verified by the theoretical analysis. The
experiments on the tasks, including pair matching, canonical correlation
analysis, transfer learning, and reconstruction demonstrate the
state-of-the-art performance of BNN, which may provide new insights into the
aspect of common representation learning.
