  Supervised learning is limited both by the quantity and quality of the
labeled data. In the field of medical record tagging, writing styles between
hospitals vary drastically. The knowledge learned from one hospital might not
transfer well to another. This problem is amplified in veterinary medicine
domain because veterinary clinics rarely apply medical codes to their records.
We proposed and trained the first large-scale generative modeling algorithm in
automated disease coding. We demonstrate that generative modeling can learn
discriminative features when additionally trained with supervised fine-tuning.
We systematically ablate and evaluate the effect of generative modeling on the
final system's performance. We compare the performance of our model with
several baselines in a challenging cross-hospital setting with substantial
domain shift. We outperform competitive baselines by a large margin. In
addition, we provide interpretation for what is learned by our model.
