  Learning the ability to generalize knowledge between similar contexts is
particularly important in medical imaging as data distributions can shift
substantially from one hospital to another, or even from one machine to
another. To strengthen generalization, most state-of-the-art techniques inject
knowledge of the data distribution shifts by enforcing constraints on learned
features or regularizing parameters. We offer an alternative approach: Learning
from Privileged Medical Imaging Information (LPMII). We show that using some
privileged information such as tumor shape or location leads to stronger domain
generalization ability than current state-of-the-art techniques. This paper
demonstrates that by using privileged information to predict the severity of
intra-layer retinal fluid in optical coherence tomography scans, the
classification accuracy of a deep learning model operating on
out-of-distribution data improves from $0.911$ to $0.934$. This paper provides
a strong starting point for using privileged information in other medical
problems requiring generalization.
