  A key goal in mechanistic interpretability is circuit analysis: finding
sparse subgraphs of models corresponding to specific behaviors or capabilities.
However, MLP sublayers make fine-grained circuit analysis on transformer-based
language models difficult. In particular, interpretable features -- such as
those found by sparse autoencoders (SAEs) -- are typically linear combinations
of extremely many neurons, each with its own nonlinearity to account for.
Circuit analysis in this setting thus either yields intractably large circuits
or fails to disentangle local and global behavior. To address this we explore
transcoders, which seek to faithfully approximate a densely activating MLP
layer with a wider, sparsely-activating MLP layer. We successfully train
transcoders on language models with 120M, 410M, and 1.4B parameters, and find
them to perform at least on par with SAEs in terms of sparsity, faithfulness,
and human-interpretability. We then introduce a novel method for using
transcoders to perform weights-based circuit analysis through MLP sublayers.
The resulting circuits neatly factorize into input-dependent and
input-invariant terms. Finally, we apply transcoders to reverse-engineer
unknown circuits in the model, and we obtain novel insights regarding the
greater-than circuit in GPT2-small. Our results suggest that transcoders can
prove effective in decomposing model computations involving MLPs into
interpretable circuits. Code is available at
https://github.com/jacobdunefsky/transcoder_circuits.
