  Many real-world tasks such as classification of digital histopathology images
and 3D object detection involve learning from a set of instances. In these
cases, only a group of instances or a set, collectively, contains meaningful
information and therefore only the sets have labels, and not individual data
instances. In this work, we present a permutation invariant neural network
called Memory-based Exchangeable Model (MEM) for learning set functions. The
MEM model consists of memory units that embed an input sequence to high-level
features enabling the model to learn inter-dependencies among instances through
a self-attention mechanism. We evaluated the learning ability of MEM on various
toy datasets, point cloud classification, and classification of lung whole
slide images (WSIs) into two subtypes of lung cancer---Lung Adenocarcinoma, and
Lung Squamous Cell Carcinoma. We systematically extracted patches from lung
WSIs downloaded from The Cancer Genome Atlas~(TCGA) dataset, the largest public
repository of WSIs, achieving a competitive accuracy of 84.84\% for
classification of two sub-types of lung cancer. The results on other datasets
are promising as well, and demonstrate the efficacy of our model.
