  Encoder-decoder architectures are prominent building blocks of
state-of-the-art solutions for tasks across multiple fields where deep learning
(DL) or foundation models play a key role. Although there is a growing
community working on the provision of interpretation for DL models as well as
considerable work in the neuro-symbolic community seeking to integrate symbolic
representations and DL, many open questions remain around the need for better
tools for visualization of the inner workings of DL architectures. In
particular, encoder-decoder models offer an exciting opportunity for
visualization and editing by humans of the knowledge implicitly represented in
model weights. In this work, we explore ways to create an abstraction for
segments of the network as a two-way graph-based representation. Changes to
this graph structure should be reflected directly in the underlying tensor
representations. Such two-way graph representation enables new neuro-symbolic
systems by leveraging the pattern recognition capabilities of the
encoder-decoder along with symbolic reasoning carried out on the graphs. The
approach is expected to produce new ways of interacting with DL models but also
to improve performance as a result of the combination of learning and reasoning
capabilities.
