  We present an approach to learning multi-sense word embeddings relying both
on monolingual and bilingual information. Our model consists of an encoder,
which uses monolingual and bilingual context (i.e. a parallel sentence) to
choose a sense for a given word, and a decoder which predicts context words
based on the chosen sense. The two components are estimated jointly. We observe
that the word representations induced from bilingual data outperform the
monolingual counterparts across a range of evaluation tasks, even though
crosslingual information is not available at test time.
