  Clinical documentation is an important aspect of clinicians' daily work and
often demands a significant amount of time. The BioNLP 2024 Shared Task on
Streamlining Discharge Documentation (Discharge Me!) aims to alleviate this
documentation burden by automatically generating discharge summary sections,
including brief hospital course and discharge instruction, which are often
time-consuming to synthesize and write manually. We approach the generation
task by fine-tuning multiple open-sourced language models (LMs), including both
decoder-only and encoder-decoder LMs, with various configurations on input
context. We also examine different setups for decoding algorithms, model
ensembling or merging, and model specialization. Our results show that
conditioning on the content of discharge summary prior to the target sections
is effective for the generation task. Furthermore, we find that smaller
encoder-decoder LMs can work as well or even slightly better than larger
decoder based LMs fine-tuned through LoRA. The model checkpoints from our team
(aehrc) are openly available.
