  Existing X-ray based pre-trained vision models are usually conducted on a
relatively small-scale dataset (less than 500k samples) with limited resolution
(e.g., 224 $\times$ 224). However, the key to the success of self-supervised
pre-training large models lies in massive training data, and maintaining high
resolution in the field of X-ray images is the guarantee of effective solutions
to difficult miscellaneous diseases. In this paper, we address these issues by
proposing the first high-definition (1280 $\times$ 1280) X-ray based
pre-trained foundation vision model on our newly collected large-scale dataset
which contains more than 1 million X-ray images. Our model follows the masked
auto-encoder framework which takes the tokens after mask processing (with a
high rate) is used as input, and the masked image patches are reconstructed by
the Transformer encoder-decoder network. More importantly, we introduce a novel
context-aware masking strategy that utilizes the chest contour as a boundary
for adaptive masking operations. We validate the effectiveness of our model on
two downstream tasks, including X-ray report generation and disease
recognition. Extensive experiments demonstrate that our pre-trained medical
foundation vision model achieves comparable or even new state-of-the-art
performance on downstream benchmark datasets. The source code and pre-trained
models of this paper will be released on
https://github.com/Event-AHU/Medical_Image_Analysis.
