  Learning concepts from natural high-dimensional data (e.g., images) holds
potential in building human-aligned and interpretable machine learning models.
Despite its encouraging prospect, formalization and theoretical insights into
this crucial task are still lacking. In this work, we formalize concepts as
discrete latent causal variables that are related via a hierarchical causal
model that encodes different abstraction levels of concepts embedded in
high-dimensional data (e.g., a dog breed and its eye shapes in natural images).
We formulate conditions to facilitate the identification of the proposed causal
model, which reveals when learning such concepts from unsupervised data is
possible. Our conditions permit complex causal hierarchical structures beyond
latent trees and multi-level directed acyclic graphs in prior work and can
handle high-dimensional, continuous observed variables, which is well-suited
for unstructured data modalities such as images. We substantiate our
theoretical claims with synthetic data experiments. Further, we discuss our
theory's implications for understanding the underlying mechanisms of latent
diffusion models and provide corresponding empirical evidence for our
theoretical insights.
