  The main alternatives nowadays to deal with sequences are Recurrent Neural
Networks (RNN), Convolutional Neural Networks (CNN) architectures and the
Transformer. In this context, RNN's, CNN's and Transformer have most commonly
been used as an encoder-decoder architecture with multiple layers in each
module. Far beyond this, these architectures are the basis for the contextual
word embeddings which are revolutionizing most natural language downstream
applications. However, intermediate layer representations in sequence-based
architectures can be difficult to interpret. To make each layer representation
within these architectures more accessible and meaningful, we introduce a
web-based tool that visualizes them both at the sentence and token level. We
present three use cases. The first analyses gender issues in contextual word
embeddings. The second and third are showing multilingual intermediate
representations for sentences and tokens and the evolution of these
intermediate representations along the multiple layers of the decoder and in
the context of multilingual machine translation.
