  In this paper, we introduce a new nonlinear optical channel equalizer based
on Transformers. By leveraging parallel computation and attending directly to
the memory across a sequence of symbols, we show that Transformers can be used
effectively for nonlinear compensation (NLC) in coherent long-haul transmission
systems. For this application, we present an implementation of the encoder part
of the Transformer and analyze its performance over a wide range of different
hyper-parameters. It is shown that by proper embeddings and processing blocks
of symbols at each iteration and also carefully selecting subsets of the
encoder's output to be processed together, an efficient nonlinear equalization
can be achieved for different complexity constraints. To reduce the
computational complexity of the attention mechanism, we further propose the use
of a physic-informed mask inspired by nonlinear perturbation theory. We also
compare the Transformer-NLC with digital back-propagation (DBP) under different
transmission scenarios in order to demonstrate the flexibility and
generalizability of the proposed data-driven solution.
