  Inspired by the remarkable zero-shot generalization capacity of
vision-language pre-trained model, we seek to leverage the supervision from
CLIP model to alleviate the burden of data labeling. However, such supervision
inevitably contains the label noise, which significantly degrades the
discriminative power of the classification model. In this work, we propose
Transductive CLIP, a novel framework for learning a classification network with
noisy labels from scratch. Firstly, a class-conditional contrastive learning
mechanism is proposed to mitigate the reliance on pseudo labels and boost the
tolerance to noisy labels. Secondly, ensemble labels is adopted as a pseudo
label updating strategy to stabilize the training of deep neural networks with
noisy labels. This framework can reduce the impact of noisy labels from CLIP
model effectively by combining both techniques. Experiments on multiple
benchmark datasets demonstrate the substantial improvements over other
state-of-the-art methods.
