  Explainable Artificial Intelligence (XAI) poses a significant challenge in
providing transparent and understandable insights into complex AI models.
Traditional post-hoc algorithms, while useful, often struggle to deliver
interpretable explanations. Concept-based models offer a promising avenue by
incorporating explicit representations of concepts to enhance interpretability.
However, existing research on automatic concept discovery methods is often
limited by lower-level concepts, costly human annotation requirements, and a
restricted domain of background knowledge. In this study, we explore the
potential of a Large Language Model (LLM), specifically GPT-4, by leveraging
its domain knowledge and common-sense capability to generate high-level
concepts that are meaningful as explanations for humans, for a specific setting
of image classification. We use minimal textual object information available in
the data via prompting to facilitate this process. To evaluate the output, we
compare the concepts generated by the LLM with two other methods: concepts
generated by humans and the ECII heuristic concept induction system. Since
there is no established metric to determine the human understandability of
concepts, we conducted a human study to assess the effectiveness of the
LLM-generated concepts. Our findings indicate that while human-generated
explanations remain superior, concepts derived from GPT-4 are more
comprehensible to humans compared to those generated by ECII.
