  The assessment of protein expression in immunohistochemistry (IHC) images
provides important diagnostic, prognostic and predictive information for
guiding cancer diagnosis and therapy. Manual scoring of IHC images represents a
logistical challenge, as the process is labor intensive and time consuming.
Since the last decade, computational methods have been developed to enable the
application of quantitative methods for the analysis and interpretation of
protein expression in IHC images. These methods have not yet replaced manual
scoring for the assessment of IHC in the majority of diagnostic laboratories
and in many large-scale research studies. An alternative approach is
crowdsourcing the quantification of IHC images to an undefined crowd. The aim
of this study is to quantify IHC images for labeling of ER status with two
different crowdsourcing approaches, image labeling and nuclei labeling, and
compare their performance with automated methods. Crowdsourcing-derived scores
obtained greater concordance with the pathologist interpretations for both
image labeling and nuclei labeling tasks (83% and 87%), as compared to the
pathologist concordance achieved by the automated method (81%) on 5,483 TMA
images from 1,909 breast cancer patients. This analysis shows that
crowdsourcing the scoring of protein expression in IHC images is a promising
new approach for large scale cancer molecular pathology studies.
