  Context-aware translation can be achieved by processing a concatenation of
consecutive sentences with the standard Transformer architecture. This paper
investigates the intuitive idea of providing the model with explicit
information about the position of the sentences contained in the concatenation
window. We compare various methods to encode sentence positions into token
representations, including novel methods. Our results show that the Transformer
benefits from certain sentence position encoding methods on English to Russian
translation if trained with a context-discounted loss (Lupo et al., 2022).
However, the same benefits are not observed in English to German. Further
empirical efforts are necessary to define the conditions under which the
proposed approach is beneficial.
