  A growing body of work uses Natural Language Processing (NLP) methods to
automatically generate medical notes from audio recordings of doctor-patient
consultations. However, there are very few studies on how such systems could be
used in clinical practice, how clinicians would adjust to using them, or how
system design should be influenced by such considerations. In this paper, we
present three rounds of user studies, carried out in the context of developing
a medical note generation system. We present, analyse and discuss the
participating clinicians' impressions and views of how the system ought to be
adapted to be of value to them. Next, we describe a three-week test run of the
system in a live telehealth clinical practice. Major findings include (i) the
emergence of five different note-taking behaviours; (ii) the importance of the
system generating notes in real time during the consultation; and (iii) the
identification of a number of clinical use cases that could prove challenging
for automatic note generation systems.
