  Enhancing model interpretability can address spurious correlations by
revealing how models draw their predictions. Concept Bottleneck Models (CBMs)
can provide a principled way of disclosing and guiding model behaviors through
human-understandable concepts, albeit at a high cost of human efforts in data
annotation. In this paper, we leverage a synergy of multiple foundation models
to construct CBMs with nearly no human effort. We discover undesirable biases
in CBMs built on pre-trained models and propose a novel framework designed to
exploit pre-trained models while being immune to these biases, thereby reducing
vulnerability to spurious correlations. Specifically, our method offers a
seamless pipeline that adopts foundation models for assessing potential
spurious correlations in datasets, annotating concepts for images, and refining
the annotations for improved robustness. We evaluate the proposed method on
multiple datasets, and the results demonstrate its effectiveness in reducing
model reliance on spurious correlations while preserving its interpretability.
