  Learning by contrasting positive and negative samples is a general strategy
adopted by many methods. Noise contrastive estimation (NCE) for word embeddings
and translating embeddings for knowledge graphs are examples in NLP employing
this approach. In this work, we view contrastive learning as an abstraction of
all such methods and augment the negative sampler into a mixture distribution
containing an adversarially learned sampler. The resulting adaptive sampler
finds harder negative examples, which forces the main model to learn a better
representation of the data. We evaluate our proposal on learning word
embeddings, order embeddings and knowledge graph embeddings and observe both
faster convergence and improved results on multiple metrics.
