  Joint representation learning of words and entities benefits many NLP tasks,
but has not been well explored in cross-lingual settings. In this paper, we
propose a novel method for joint representation learning of cross-lingual words
and entities. It captures mutually complementary knowledge, and enables
cross-lingual inferences among knowledge bases and texts. Our method does not
require parallel corpora, and automatically generates comparable data via
distant supervision using multi-lingual knowledge bases. We utilize two types
of regularizers to align cross-lingual words and entities, and design knowledge
attention and cross-lingual attention to further reduce noises. We conducted a
series of experiments on three tasks: word translation, entity relatedness, and
cross-lingual entity linking. The results, both qualitatively and
quantitatively, demonstrate the significance of our method.
