  We propose a novel deep training algorithm for joint representation of audio
and visual information which consists of a single stream network (SSNet)
coupled with a novel loss function to learn a shared deep latent space
representation of multimodal information. The proposed framework characterizes
the shared latent space by leveraging the class centers which helps to
eliminate the need for pairwise or triplet supervision. We quantitatively and
qualitatively evaluate the proposed approach on VoxCeleb, a benchmarks
audio-visual dataset on a multitude of tasks including cross-modal
verification, cross-modal matching, and cross-modal retrieval. State-of-the-art
performance is achieved on cross-modal verification and matching while
comparable results are observed on the remaining applications. Our experiments
demonstrate the effectiveness of the technique for cross-modal biometric
applications.
