  This paper describes Charles University submission for Terminology
translation Shared Task at WMT21. The objective of this task is to design a
system which translates certain terms based on a provided terminology database,
while preserving high overall translation quality. We competed in
English-French language pair. Our approach is based on providing the desired
translations alongside the input sentence and training the model to use these
provided terms. We lemmatize the terms both during the training and inference,
to allow the model to learn how to produce correct surface forms of the words,
when they differ from the forms provided in the terminology database. Our
submission ranked second in Exact Match metric which evaluates the ability of
the model to produce desired terms in the translation.
