  In the last few years, neural networks have been intensively used to develop
meaningful distributed representations of words and contexts around them. When
these representations, also known as "embeddings", are learned from
unsupervised large corpora, they can be transferred to different tasks with
positive effects in terms of performances, especially when only a few
supervisions are available. In this work, we further extend this concept, and
we present an unsupervised neural architecture that jointly learns word and
context embeddings, processing words as sequences of characters. This allows
our model to spot the regularities that are due to the word morphology, and to
avoid the need of a fixed-sized input vocabulary of words. We show that we can
learn compact encoders that, despite the relatively small number of parameters,
reach high-level performances in downstream tasks, comparing them with related
state-of-the-art approaches or with fully supervised methods.
