  We address the challenging problem of whole slide image (WSI) classification.
WSIs have very high resolutions and usually lack localized annotations. WSI
classification can be cast as a multiple instance learning (MIL) problem when
only slide-level labels are available. We propose a MIL-based method for WSI
classification and tumor detection that does not require localized annotations.
Our method has three major components. First, we introduce a novel MIL
aggregator that models the relations of the instances in a dual-stream
architecture with trainable distance measurement. Second, since WSIs can
produce large or unbalanced bags that hinder the training of MIL models, we
propose to use self-supervised contrastive learning to extract good
representations for MIL and alleviate the issue of prohibitive memory cost for
large bags. Third, we adopt a pyramidal fusion mechanism for multiscale WSI
features, and further improve the accuracy of classification and localization.
Our model is evaluated on two representative WSI datasets. The classification
accuracy of our model compares favorably to fully-supervised methods, with less
than 2% accuracy gap across datasets. Our results also outperform all previous
MIL-based methods. Additional benchmark results on standard MIL datasets
further demonstrate the superior performance of our MIL aggregator on general
MIL problems. GitHub repository: https://github.com/binli123/dsmil-wsi
