  We introduce polyglot language models, recurrent neural network models
trained to predict symbol sequences in many different languages using shared
representations of symbols and conditioning on typological information about
the language to be predicted. We apply these to the problem of modeling phone
sequences---a domain in which universal symbol inventories and
cross-linguistically shared feature representations are a natural fit.
Intrinsic evaluation on held-out perplexity, qualitative analysis of the
learned representations, and extrinsic evaluation in two downstream
applications that make use of phonetic features show (i) that polyglot models
better generalize to held-out data than comparable monolingual models and (ii)
that polyglot phonetic feature representations are of higher quality than those
learned monolingually.
