  In recent years, the softmax model and its fast approximations have become
the de-facto loss functions for deep neural networks when dealing with
multi-class prediction. This loss has been extended to language modeling and
recommendation, two fields that fall into the framework of learning from
Positive and Unlabeled data. In this paper, we stress the different drawbacks
of the current family of softmax losses and sampling schemes when applied in a
Positive and Unlabeled learning setup. We propose both a Relaxed Softmax loss
(RS) and a new negative sampling scheme based on Boltzmann formulation. We show
that the new training objective is better suited for the tasks of density
estimation, item similarity and next-event prediction by driving uplifts in
performance on textual and recommendation datasets against classical softmax.
