  Nested pairwise frames is a method for relative benchmarking of cell or
tissue digital pathology models against manual pathologist annotations on a set
of sampled patches. At a high level, the method compares agreement between a
candidate model and pathologist annotations with agreement among pathologists'
annotations. This evaluation framework addresses fundamental issues of data
size and annotator variability in using manual pathologist annotations as a
source of ground truth for model validation. We implemented nested pairwise
frames evaluation for tissue classification, cell classification, and cell
count prediction tasks and show results for cell and tissue models deployed on
an H&E-stained melanoma dataset.
