  The field of natural language understanding has experienced exponential
progress in the last few years, with impressive results in several tasks. This
success has motivated researchers to study the underlying knowledge encoded by
these models. Despite this, attempts to understand their semantic capabilities
have not been successful, often leading to non-conclusive, or contradictory
conclusions among different works. Via a probing classifier, we extract the
underlying knowledge graph of nine of the most influential language models of
the last years, including word embeddings, text generators, and context
encoders. This probe is based on concept relatedness, grounded on WordNet. Our
results reveal that all the models encode this knowledge, but suffer from
several inaccuracies. Furthermore, we show that the different architectures and
training strategies lead to different model biases. We conduct a systematic
evaluation to discover specific factors that explain why some concepts are
challenging. We hope our insights will motivate the development of models that
capture concepts more precisely.
