  One-class collaborative filtering (OC-CF) is a common class of recommendation
problem where only the positive class is explicitly observed (e.g., purchases,
clicks). Autoencoder based recommenders such as AutoRec and variants
demonstrate strong performance on many OC-CF benchmarks, but also empirically
suffer from a strong popularity bias. While a careful choice of negative
samples in the OC-CF setting can mitigate popularity bias, Negative Sampling
(NS) is often better for training embeddings than for the end task itself. To
address this, we propose a two-headed AutoRec to first train an embedding layer
via one head using Negative Sampling then to train for the final task via the
second head. While this NS-AutoRec improves results for AutoRec and outperforms
many state-of-the-art baselines on OC-CF problems, we notice that Negative
Sampling can still take a large amount of time to train. Since Negative
Sampling is known to be a special case of Noise Contrastive Estimation (NCE),
we adapt a recently proposed closed-form NCE solution for collaborative
filtering to AutoRec yielding NCE-AutoRec. Overall, we show that our novel
two-headed AutoRec models (NCE-AutoRec and NS-AutoRec) successfully mitigate
the popularity bias issue and maintain competitive performance in comparison to
state-of-the-art recommenders on multiple real-world datasets.
