  The absence of adequately sufficient expert-level tumor annotations hinders
the effectiveness of supervised learning based opportunistic cancer screening
on medical imaging. Clinical reports (that are rich in descriptive textual
details) can offer a "free lunch'' supervision information and provide tumor
location as a type of weak label to cope with screening tasks, thus saving
human labeling workloads, if properly leveraged. However, predicting cancer
only using such weak labels can be very changeling since tumors are usually
presented in small anatomical regions compared to the whole 3D medical scans.
Weakly semi-supervised learning (WSSL) utilizes a limited set of voxel-level
tumor annotations and incorporates alongside a substantial number of medical
images that have only off-the-shelf clinical reports, which may strike a good
balance between minimizing expert annotation workload and optimizing screening
efficacy. In this paper, we propose a novel text-guided learning method to
achieve highly accurate cancer detection results. Through integrating
diagnostic and tumor location text prompts into the text encoder of a
vision-language model (VLM), optimization of weakly supervised learning can be
effectively performed in the latent space of VLM, thereby enhancing the
stability of training. Our approach can leverage clinical knowledge by
large-scale pre-trained VLM to enhance generalization ability, and produce
reliable pseudo tumor masks to improve cancer detection. Our extensive
quantitative experimental results on a large-scale cancer dataset, including
1,651 unique patients, validate that our approach can reduce human annotation
efforts by at least 70% while maintaining comparable cancer detection accuracy
to competing fully supervised methods (AUC value 0.961 versus 0.966).
