  The development of open-source, multilingual medical language models can
benefit a wide, linguistically diverse audience from different regions. To
promote this domain, we present contributions from the following: First, we
construct a multilingual medical corpus, containing approximately 25.5B tokens
encompassing 6 main languages, termed as MMedC, enabling auto-regressive domain
adaptation for general LLMs; Second, to monitor the development of multilingual
medical LLMs, we propose a multilingual medical multi-choice question-answering
benchmark with rationale, termed as MMedBench; Third, we have assessed a number
of open-source large language models (LLMs) on our benchmark, along with those
further auto-regressive trained on MMedC. Our final model, MMed-Llama 3, with
only 8B parameters, achieves superior performance compared to all other
open-source models on both MMedBench and English benchmarks, even rivaling
GPT-4. In conclusion, in this work, we present a large-scale corpus, a
benchmark and a series of models to support the development of multilingual
medical LLMs.
