  Deep learning approaches applied to medical imaging have reached near-human
or better-than-human performance on many diagnostic tasks. For instance, the
CheXpert competition on detecting pathologies in chest x-rays has shown
excellent multi-class classification performance. However, training and
validating deep learning models require extensive collections of images and
still produce false inferences, as identified by a human-in-the-loop. In this
paper, we introduce a practical approach to improve the predictions of a
pre-trained model through Few-Shot Learning (FSL). After training and
validating a model, a small number of false inference images are collected to
retrain the model using \textbf{\textit{Image Triplets}} - a false positive or
false negative, a true positive, and a true negative. The retrained FSL model
produces considerable gains in performance with only a few epochs and few
images. In addition, FSL opens rapid retraining opportunities for
human-in-the-loop systems, where a radiologist can relabel false inferences,
and the model can be quickly retrained. We compare our retrained model
performance with existing FSL approaches in medical imaging that train and
evaluate models at once.
