  Named-entities are inherently multilingual, and annotations in any given
language may be limited. This motivates us to consider polyglot named-entity
recognition (NER), where one model is trained using annotated data drawn from
more than one language. However, a straightforward implementation of this
simple idea does not always work in practice: naive training of NER models
using annotated data drawn from multiple languages consistently underperforms
models trained on monolingual data alone, despite having access to more
training data. The starting point of this paper is a simple solution to this
problem, in which polyglot models are fine-tuned on monolingual data to
consistently and significantly outperform their monolingual counterparts. To
explain this phenomena, we explore the sources of multilingual transfer in
polyglot NER models and examine the weight structure of polyglot models
compared to their monolingual counterparts. We find that polyglot models
efficiently share many parameters across languages and that fine-tuning may
utilize a large number of those parameters.
