  Recent advancements in pre-trained large foundation models (LFM) have yielded
significant breakthroughs across various domains, including natural language
processing and computer vision. These models have been particularly impactful
in the domain of medical diagnostic tasks. With abundant unlabeled data, an LFM
has been developed for fundus images using the Vision Transformer (VIT) and a
self-supervised learning framework. This LFM has shown promising performance in
fundus disease diagnosis across multiple datasets. On the other hand, deep
learning models have long been challenged by dataset quality issues, such as
image quality and dataset bias. To investigate the influence of data quality on
LFM, we conducted explorations in two fundus diagnosis tasks using datasets of
varying quality. Specifically, we explored the following questions: Is LFM more
robust to image quality? Is LFM affected by dataset bias? Can fine-tuning
techniques alleviate these effects? Our investigation found that LFM exhibits
greater resilience to dataset quality issues, including image quality and
dataset bias, compared to typical convolutional networks. Furthermore, we
discovered that overall fine-tuning is an effective adapter for LFM to mitigate
the impact of dataset quality issues.
