  Good communication is critical to good healthcare. Clinical dialogue is a
conversation between health practitioners and their patients, with the explicit
goal of obtaining and sharing medical information. This information contributes
to medical decision-making regarding the patient and plays a crucial role in
their healthcare journey. The reliance on note taking and manual scribing
processes are extremely inefficient and leads to manual transcription errors
when digitizing notes. Automatic Speech Recognition (ASR) plays a significant
role in speech-to-text applications, and can be directly used as a text
generator in conversational applications. However, recording clinical dialogue
presents a number of general and domain-specific challenges. In this paper, we
present a seq2seq learning approach for ASR transcription error correction of
clinical dialogues. We introduce a new Gastrointestinal Clinical Dialogue (GCD)
Dataset which was gathered by healthcare professionals from a NHS Inflammatory
Bowel Disease clinic and use this in a comparative study with four commercial
ASR systems. Using self-supervision strategies, we fine-tune a seq2seq model on
a mask-filling task using a domain-specific PubMed dataset which we have shared
publicly for future research. The BART model fine-tuned for mask-filling was
able to correct transcription errors and achieve lower word error rates for
three out of four commercial ASR outputs.
