  Electronic Health Record (EHR) data can be represented as discrete counts
over a high dimensional set of possible procedures, diagnoses, and medications.
Supervised topic models present an attractive option for incorporating EHR data
as features into a prediction problem: given a patient's record, we estimate a
set of latent factors that are predictive of the response variable. However,
existing methods for supervised topic modeling struggle to balance prediction
quality and coherence of the latent factors. We introduce a novel approach, the
prediction-focused topic model, that uses the supervisory signal to retain only
features that improve, or do not hinder, prediction performance. By removing
features with irrelevant signal, the topic model is able to learn
task-relevant, interpretable topics. We demonstrate on a EHR dataset and a
movie review dataset that compared to existing approaches, prediction-focused
topic models are able to learn much more coherent topics while maintaining
competitive predictions.
