  Probabilistic topic models like Latent Dirichlet Allocation (LDA) have been
previously extended to the bilingual setting. A fundamental modeling assumption
in several of these extensions is that the input corpora are in the form of
document pairs whose constituent documents share a single topic distribution.
However, this assumption is strong for comparable corpora that consist of
documents thematically similar to an extent only, which are, in turn, the most
commonly available or easy to obtain. In this paper we relax this assumption by
proposing for the paired documents to have separate, yet bound topic
distributions. % a binding mechanism between the distributions of the paired
documents. We suggest that the strength of the bound should depend on each
pair's semantic similarity. To estimate the similarity of documents that are
written in different languages we use cross-lingual word embeddings that are
learned with shallow neural networks. We evaluate the proposed binding
mechanism by extending two topic models: a bilingual adaptation of LDA that
assumes bag-of-words inputs and a model that incorporates part of the text
structure in the form of boundaries of semantically coherent segments. To
assess the performance of the novel topic models we conduct intrinsic and
extrinsic experiments on five bilingual, comparable corpora of English
documents with French, German, Italian, Spanish and Portuguese documents. The
results demonstrate the efficiency of our approach in terms of both topic
coherence measured by the normalized point-wise mutual information, and
generalization performance measured by perplexity and in terms of Mean
Reciprocal Rank in a cross-lingual document retrieval task for each of the
language pairs.
