  Causal abstraction is a promising theoretical framework for explainable
artificial intelligence that defines when an interpretable high-level causal
model is a faithful simplification of a low-level deep learning system.
However, existing causal abstraction methods have two major limitations: they
require a brute-force search over alignments between the high-level model and
the low-level one, and they presuppose that variables in the high-level model
will align with disjoint sets of neurons in the low-level one. In this paper,
we present distributed alignment search (DAS), which overcomes these
limitations. In DAS, we find the alignment between high-level and low-level
models using gradient descent rather than conducting a brute-force search, and
we allow individual neurons to play multiple distinct roles by analyzing
representations in non-standard bases-distributed representations. Our
experiments show that DAS can discover internal structure that prior approaches
miss. Overall, DAS removes previous obstacles to conducting causal abstraction
analyses and allows us to find conceptual structure in trained neural nets.
