  Background: Large language models such as ChatGPT are capable of generating
grammatically perfect and human-like text content, and a large number of
ChatGPT-generated texts have appeared on the Internet. However, medical texts
such as clinical notes and diagnoses require rigorous validation, and erroneous
medical content generated by ChatGPT could potentially lead to disinformation
that poses significant harm to healthcare and the general public.
  Objective: This research is among the first studies on responsible and
ethical AIGC (Artificial Intelligence Generated Content) in medicine. We focus
on analyzing the differences between medical texts written by human experts and
generated by ChatGPT, and designing machine learning workflows to effectively
detect and differentiate medical texts generated by ChatGPT.
  Methods: We first construct a suite of datasets containing medical texts
written by human experts and generated by ChatGPT. In the next step, we analyze
the linguistic features of these two types of content and uncover differences
in vocabulary, part-of-speech, dependency, sentiment, perplexity, etc. Finally,
we design and implement machine learning methods to detect medical text
generated by ChatGPT.
  Results: Medical texts written by humans are more concrete, more diverse, and
typically contain more useful information, while medical texts generated by
ChatGPT pay more attention to fluency and logic, and usually express general
terminologies rather than effective information specific to the context of the
problem. A BERT-based model can effectively detect medical texts generated by
ChatGPT, and the F1 exceeds 95%.
