  Despite the advances in machine learning and digital pathology, it is not yet
clear if machine learning methods can accurately predict molecular information
merely from histomorphology. In a quest to answer this question, we built a
large-scale dataset (185538 images) with reliable measurements for Ki67, ER,
PR, and HER2 statuses. The dataset is composed of mirrored images of H\&E and
corresponding images of immunohistochemistry (IHC) assays (Ki67, ER, PR, and
HER2. These images are mirrored through registration. To increase reliability,
individual pairs were inspected and discarded if artifacts were present (tissue
folding, bubbles, etc). Measurements for Ki67, ER and PR were determined by
calculating H-Score from image analysis. HER2 measurement is based on binary
classification: 0 and 1+ (IHC scores representing a negative subset) vs 3+ (IHC
score positive subset). Cases with IHC equivocal score (2+) were excluded. We
show that a standard ViT-based pipeline can achieve prediction performances
around 90% in terms of Area Under the Curve (AUC) when trained with a proper
labeling protocol. Finally, we shed light on the ability of the trained
classifiers to localize relevant regions, which encourages future work to
improve the localizations. Our proposed dataset is publicly available:
https://ihc4bc.github.io/
