  Supervised feature learning using convolutional neural networks (CNNs) can
provide concise and disease relevant representations of medical images.
However, training CNNs requires annotated image data. Annotating medical images
can be a time-consuming task and even expert annotations are subject to
substantial inter- and intra-rater variability. Assessing visual similarity of
images instead of indicating specific pathologies or estimating disease
severity could allow non-experts to participate, help uncover new patterns, and
possibly reduce rater variability. We consider the task of assessing emphysema
extent in chest CT scans. We derive visual similarity triplets from visually
assessed emphysema extent and learn a low dimensional embedding using CNNs. We
evaluate the networks on 973 images, and show that the CNNs can learn disease
relevant feature representations from derived similarity triplets. To our
knowledge this is the first medical image application where similarity triplets
has been used to learn a feature representation that can be used for embedding
unseen test images
