  Idiomatic expressions can be problematic for natural language processing
applications as their meaning cannot be inferred from their constituting words.
A lack of successful methodological approaches and sufficiently large datasets
prevents the development of machine learning approaches for detecting idioms,
especially for expressions that do not occur in the training set. We present an
approach, called MICE, that uses contextual embeddings for that purpose. We
present a new dataset of multi-word expressions with literal and idiomatic
meanings and use it to train a classifier based on two state-of-the-art
contextual word embeddings: ELMo and BERT. We show that deep neural networks
using both embeddings perform much better than existing approaches, and are
capable of detecting idiomatic word use, even for expressions that were not
present in the training set. We demonstrate cross-lingual transfer of developed
models and analyze the size of the required dataset.
