  Transformers exhibit impressive capabilities but are often regarded as black
boxes due to challenges in understanding the complex nonlinear relationships
between features. Interpreting machine learning models is of paramount
importance to mitigate risks, and mechanistic interpretability is in particular
of current interest as it opens up a window for guiding manual modifications
and reverse-engineering solutions. In this work, we introduce contextual
decomposition for transformers (CD-T), extending a prior work on CD for RNNs
and CNNs, to address mechanistic interpretation computationally efficiently.
CD-T is a flexible interpretation method for transformers. It can capture
contributions of combinations of input features or source internal components
(e.g. attention heads, feed-forward networks) to (1) final predictions or (2)
the output of any target internal component. Using CD-T, we propose a novel
algorithm for circuit discovery. On a real-world pathology report
classification task: we show CD-T distills a more faithful circuit of attention
heads with improved computational efficiency (speed up 2x) than a prior
benchmark, path patching. As a versatile interpretation method, CD-T also
exhibits exceptional capabilities for local interpretations. CD-T is shown to
reliably find words and phrases of contrasting sentiment/topic on SST-2 and
AGNews datasets. Through human experiments, we demonstrate CD-T enables users
to identify the more accurate of two models and to better trust a model's
outputs compared to alternative interpretation methods such as SHAP and LIME.
