  Chest X-Ray imaging is one of the most common radiological tools for
detection of various pathologies related to the chest area and lung function.
In a clinical setting, automated assessment of chest radiographs has the
potential of assisting physicians in their decision making process and optimize
clinical workflows, for example by prioritizing emergency patients.
  Most work analyzing the potential of machine learning models to classify
chest X-ray images focuses on vision methods processing and predicting
pathologies for one image at a time. However, many patients undergo such a
procedure multiple times during course of a treatment or during a single
hospital stay. The patient history, that is previous images and especially the
corresponding diagnosis contain useful information that can aid a
classification system in its prediction.
  In this study, we analyze how information about diagnosis can improve
CNN-based image classification models by constructing a novel dataset from the
well studied CheXpert dataset of chest X-rays. We show that a model trained on
additional patient history information outperforms a model trained without the
information by a significant margin.
  We provide code to replicate the dataset creation and model training.
