  In this work we explored building automatic speech recognition models for
transcribing doctor patient conversation. We collected a large scale dataset of
clinical conversations ($14,000$ hr), designed the task to represent the real
word scenario, and explored several alignment approaches to iteratively improve
data quality. We explored both CTC and LAS systems for building speech
recognition models. The LAS was more resilient to noisy data and CTC required
more data clean up. A detailed analysis is provided for understanding the
performance for clinical tasks. Our analysis showed the speech recognition
models performed well on important medical utterances, while errors occurred in
causal conversations. Overall we believe the resulting models can provide
reasonable quality in practice.
