  In this paper we tackle multilingual named entity recognition task. We use
the BERT Language Model as embeddings with bidirectional recurrent network,
attention, and NCRF on the top. We apply multilingual BERT only as embedder
without any fine-tuning. We test out model on the dataset of the BSNLP shared
task, which consists of texts in Bulgarian, Czech, Polish and Russian
languages.
