  Despite their success, Large-Language Models (LLMs) still face criticism as
their lack of interpretability limits their controllability and reliability.
Traditional post-hoc interpretation methods, based on attention and
gradient-based analysis, offer limited insight into the model's decision-making
processes. In the image field, Concept-based models have emerged as
explainable-by-design architectures, employing human-interpretable features as
intermediate representations. However, these methods have not been yet adapted
to textual data, mainly because they require expensive concept annotations,
which are impractical for real-world text data. This paper addresses this
challenge by proposing a self-supervised Interpretable Concept Embedding Models
(ICEMs). We leverage the generalization abilities of LLMs to predict the
concepts labels in a self-supervised way, while we deliver the final
predictions with an interpretable function. The results of our experiments show
that ICEMs can be trained in a self-supervised way achieving similar
performance to fully supervised concept-based models and end-to-end black-box
ones. Additionally, we show that our models are (i) interpretable, offering
meaningful logical explanations for their predictions; (ii) interactable,
allowing humans to modify intermediate predictions through concept
interventions; and (iii) controllable, guiding the LLMs' decoding process to
follow a required decision-making path.
