  Representation learning approaches require a massive amount of discriminative
training data, which is unavailable in many scenarios, such as healthcare,
smart city, education, etc. In practice, people refer to crowdsourcing to get
annotated labels. However, due to issues like data privacy, budget limitation,
shortage of domain-specific annotators, the number of crowdsourced labels is
still very limited. Moreover, because of annotators' diverse expertise,
crowdsourced labels are often inconsistent. Thus, directly applying existing
supervised representation learning (SRL) algorithms may easily get the
overfitting problem and yield suboptimal solutions. In this paper, we propose
\emph{NeuCrowd}, a unified framework for SRL from crowdsourced labels. The
proposed framework (1) creates a sufficient number of high-quality
\emph{n}-tuplet training samples by utilizing safety-aware sampling and robust
anchor generation; and (2) automatically learns a neural sampling network that
adaptively learns to select effective samples for SRL networks. The proposed
framework is evaluated on both one synthetic and three real-world data sets.
The results show that our approach outperforms a wide range of state-of-the-art
baselines in terms of prediction accuracy and AUC. To encourage reproducible
results, we make our code publicly available at
\url{https://github.com/tal-ai/NeuCrowd_KAIS2021}.
