  Crowd sourcing has become a widely adopted scheme to collect ground truth
labels. However, it is a well-known problem that these labels can be very
noisy. In this paper, we demonstrate how to learn a deep convolutional neural
network (DCNN) from noisy labels, using facial expression recognition as an
example. More specifically, we have 10 taggers to label each input image, and
compare four different approaches to utilizing the multiple labels: majority
voting, multi-label learning, probabilistic label drawing, and cross-entropy
loss. We show that the traditional majority voting scheme does not perform as
well as the last two approaches that fully leverage the label distribution. An
enhanced FER+ data set with multiple labels for each face image will also be
shared with the research community.
