  Building meaningful phrase representations is challenging because phrase
meanings are not simply the sum of their constituent meanings. Lexical
composition can shift the meanings of the constituent words and introduce
implicit information. We tested a broad range of textual representations for
their capacity to address these issues. We found that as expected,
contextualized word representations perform better than static word embeddings,
more so on detecting meaning shift than in recovering implicit information, in
which their performance is still far from that of humans. Our evaluation suite,
including 5 tasks related to lexical composition effects, can serve future
research aiming to improve such representations.
