  Large modern surveys require efficient review of data in order to find
transient sources such as supernovae, and to distinguish such sources from
artefacts and noise. Much effort has been put into the development of automatic
algorithms, but surveys still rely on human review of targets. This paper
presents an integrated system for the identification of supernovae in data from
Pan-STARRS1, combining classifications from volunteers participating in a
citizen science project with those from a convolutional neural network. The
unique aspect of this work is the deployment, in combination, of both human and
machine classifications for near real-time discovery in an astronomical
project. We show that the combination of the two methods outperforms either one
used individually. This result has important implications for the future
development of transient searches, especially in the era of LSST and other
large-throughput surveys.
