  The need for improved diagnostic methods in ophthalmology is acute,
especially in the less developed regions with limited access to specialists and
advanced equipment. Therefore, we introduce VisionUnite, a novel
vision-language foundation model for ophthalmology enhanced with clinical
knowledge. VisionUnite has been pretrained on an extensive dataset comprising
1.24 million image-text pairs, and further refined using our proposed MMFundus
dataset, which includes 296,379 high-quality fundus image-text pairs and
889,137 simulated doctor-patient dialogue instances. Our experiments indicate
that VisionUnite outperforms existing generative foundation models such as
GPT-4V and Gemini Pro. It also demonstrates diagnostic capabilities comparable
to junior ophthalmologists. VisionUnite performs well in various clinical
scenarios including open-ended multi-disease diagnosis, clinical explanation,
and patient interaction, making it a highly versatile tool for initial
ophthalmic disease screening. VisionUnite can also serve as an educational aid
for junior ophthalmologists, accelerating their acquisition of knowledge
regarding both common and rare ophthalmic conditions. VisionUnite represents a
significant advancement in ophthalmology, with broad implications for
diagnostics, medical education, and understanding of disease mechanisms.
